Module 08 · AI Fundamentals Lab

If the model was never trained on your documents, where does the answer come from?

You will search a fictional company’s security handbook, watch the retriever score all 14 documents for real, and read the exact prompt it builds. Then you will break it three different ways and see that the model was never the problem.

Three ways to make a model know your stuff

People reach for these interchangeably, and they are not interchangeable. Only one of them actually changes the model.

RAG

Change the prompt

What it does: searches your documents at question time and pastes the best matches in above the question.

Update a policy: edit the document. The next question picks it up. No retraining.

Cost: a search plus a longer prompt on every single question, forever.

Fails when: the search misses. Then the model is answering from nothing.

Fine-tuning

Change the model

What it does: adjusts the model’s weights on your examples. Good for teaching a style, a format or a task — much less good for teaching facts.

Update a policy: collect data, train again, test again, redeploy. Days, not minutes.

Cost: paid up front, then cheap per question.

Fails when: facts change, or you need to know which document an answer came from. A fine-tuned model cannot cite anything.

Long context

Paste everything

What it does: skips the search and puts the whole handbook in the prompt.

Update a policy: edit the document, same as RAG.

Cost: you pay for every page on every question, and long prompts get slower and lose things in the middle (see module 03).

Fails when: your documents outgrow the window, which for a real company is immediately. 14 chunks fit. 14,000 do not.

Why the security team usually prefers RAG. Fine-tuning bakes your documents into the weights: you cannot take them back out, you cannot tell what the model absorbed, and anyone who can query the model can potentially pull fragments of it back. With retrieval the documents stay in your own store, behind your own access control. Revoke someone’s access and the next retrieval simply cannot see those chunks. Every answer names the chunks it used, so an auditor can ask “which document said that?” and get a real answer. Two habits follow from this: retrieve as the user, not as a service account, and log every retrieval — the chunk list is the audit trail.

The comparison on this page is written by hand, like the rest of the prose here. Only the retrieval in the other tabs is computed.

RAG changes what is in the prompt, not what the model knows — so a wrong answer is usually a retrieval failure.

Before you blame the model, read the prompt it was given. If the fact was not in there, no model on earth could have answered, and the fix is in the search, the chunking or the documents.

My observations

Kept in this browser tab only, for this session.

What surprised you?

Where would retrieval fail on your own company’s documents?