What Is RAG?
Reviewed by Human · Updated October 30, 2026
The model answering your RAG query may not have known a single fact in that answer thirty seconds ago. It looked nothing up on its own.
Real world
An open-book exam. The student doesn't memorize the textbook — they're handed the relevant pages and told to answer using only those pages.
AI parallel
RAG hands the model the relevant pages (retrieved chunks) at question time. A good prompt tells the model to answer from the pages, cite which page, and say 'not in the pages' when the answer isn't there — exactly the instructions a strict proctor would give.
What Is Retrieval-Augmented Generation?
**Retrieval-Augmented Generation (RAG)**: an architecture that retrieves relevant documents from an external source, then inserts them into the prompt so the model answers from that supplied context instead of from memory alone. The pipeline is three stages: 1. **Retrieve** — a search step (usually vector similarity) pulls the top-k chunks matching the user's query. 2. **Augment** — those chunks are formatted and injected into the prompt as **context**. 3. **Generate** — the model produces an answer conditioned on that context. **Chunk**: a small slice of a source document — a paragraph or a few hundred tokens — that retrieval treats as one unit. You usually don't control stage 1 (that's the retrieval engineer's job). You own stage 2's *phrasing* and all of stage 3's *instructions*. That's where prompt quality lives.
0
Counterintuitive but load-bearing: handing a model correct documents does not stop it from inventing answers. Without an explicit grounding instruction, a model will happily blend retrieved facts with confident guesses from its training data — and present both in the same fluent voice. Retrieval supplies the truth; your prompt is what forces the model to actually use it. Modules 4 and 5 are built entirely around that fact.
Your turn
Below is a bare RAG prompt — context gets pasted in, then a question. It's missing the one instruction that makes RAG behave like an open-book exam instead of a memory quiz. Add a single sentence that constrains the model to the supplied context.
Reflect
Compare your sentence to: 'Answer using only the context above. If the answer is not present in the context, reply: "Not found in the provided sources."' Did yours give the model an explicit escape hatch for missing information? That escape hatch is the difference between grounded and hallucinated.
In a RAG system, where do the facts in a good answer primarily come from?
Go deeper
You've finished this module.
Mark it complete to earn your XP and keep your streak alive.
Progress saved locally · Sign up to earn XP