RAG (Retrieval-Augmented Generation)
RAG is an approach where the model first retrieves relevant fragments from your own data and answers from them. It is how a model works with fresh or private material it was never trained on, and it invents markedly less.
How it works
Documents are chunked in advance and turned into vectors, numeric representations of meaning, which go into a vector database. A query is vectorised too, the database finds the closest fragments by meaning, and they are placed into the context alongside the question.
The model then answers from that material rather than from memory. Two gains follow: the answer rests on a checkable source, and it can carry a link to the specific document.
Retrieval works on meaning rather than words. A query about "sending something back" finds the "returns policy" section even without matching words, because vectors are compared rather than strings.
RAG or fine-tuning
- RAG when the data changesDocuments update without retraining the model; reindexing the store is enough. Fine-tuning cannot do that.
- Fine-tuning when the behaviour changesAnswer style, output format, a specific jargon: that is fine-tuning territory, not retrieval.
- Quality lives in the chunkingChunks too small lose context, chunks too large blur retrieval. This is the main knob in the whole design.
- Bad retrieval, bad answerIf the store returns the wrong fragments the model will answer confidently from them. RAG reduces invention; it does not remove the need to check.