05 · AI Fundamentals Lab
Which model should you actually use?
Send one prompt to five very different kinds of model and compare the answers, the speed, the cost and where each one can run. You will see the cheap model do a job perfectly well — and then watch it fall apart on a job one task later.
1 · Pick a task
The same prompt goes to all five archetypes. Change the task and watch which columns hold up.
2 · The prompt
3 · Five answers
Illustrative archetype answers, written to show characteristic differences. Not live model output, and not quoted from any product.
Five real situations
Pick the situation, then pick the model you would reach for. There is no single right answer here — the point is to see what is really driving the decision. Usually it is three things: how many times you will run it, how sensitive the data is, and what happens when the answer is wrong.
What actually differs between models
“Bigger” is only one of five things that change. The other four decide more of your day-to-day than size does.
Parameters, or “size”
A model is a very large pile of numbers, learned during training. The count of those numbers is its parameter count — a billion, eight billion, hundreds of billions. More parameters means more room to store patterns, which mostly shows up as better multi-step reasoning and better recall of rare facts. It also means more hardware, more electricity, more waiting, and more money per request. Size is a dial with a cost on the other end, not a quality score.
Training data and cutoff
Every model was trained on text collected up to some date and knows nothing that happened after it. A model trained before a vulnerability was published cannot know about it, and will often invent something plausible rather than say so. Two models of the same size can differ a lot simply because one was fed more code, or more security writing, or more of one language.
Context length
How much text the model can hold in front of it at once — your prompt, the conversation so far, and any documents you pasted in. A short context means a long log file simply will not fit. This is module 03’s subject, and it is often the thing that rules a model out before capability ever comes up.
Specialisation: base, instruct, code
A base model just continues text; it was never taught to follow an instruction, so asking it a question gets you more questions. An instruct (or chat) model has had extra training to answer what you asked — this is what you normally want. A code model has been trained heavily on source code and often matches a far larger general model inside that speciality, while being noticeably worse outside it. Specialisation is a real lever: a small specialist can beat a big generalist on its home turf for a fraction of the cost.
Open-weight or hosted — the one that matters for security
Open-weight models are files you can download and run on hardware you control. Nothing you type leaves your network. You can run them air-gapped, on a machine with no route to the internet at all.
Hosted models live behind somebody else’s API. To use one you send them your prompt — and your prompt is your data. Contracts and settings may say the provider will not train on it or will delete it, and that may well be enough for your risk appetite. But that is a contractual control, not a technical one. If a rule says the data never leaves the building, only running the weights yourself makes that statement true.
The five archetypes side by side
Rough, illustrative shapes — real models vary enormously within every row.
| Archetype | Rough size | Where it can run | What you trade away |
|---|---|---|---|
| Small fast model | ~1 billion parameters | A laptop, sometimes a phone. Fully air-gappable. | Multi-step reasoning, rare facts, long context. |
| Mid-size general model | ~8 billion parameters | One decent GPU in your own rack. Air-gappable. | The hardest reasoning; some long-tail knowledge. |
| Large frontier-class model | Very large; often undisclosed | Usually a hosted API. Some are open-weight but need serious hardware. | Money, seconds of latency, and often data locality. |
| Code-specialised model | ~7 billion parameters | One decent GPU. Air-gappable. | General conversation, tone, and non-code reasoning. |
| Older-generation model | Whatever was large a few years ago | Depends entirely on the model. | Instruction-following, recent knowledge, price per answer. |
Pick the smallest model that reliably does the job — capability you do not need is just latency, cost and exposure.
“Reliably” is doing real work in that sentence. You only know whether the small one clears the bar by testing it on your own task, with your own examples, and looking at what it gets wrong.
My observations
Kept in this browser tab only, for this session.