Loading prices...
All news
Glossy orange filing cabinet drawer pulled open with white index cards flying in an arc into a violet and silver processor chip, black-violet background with violet glow and cyan sparks

RAG vs fine-tuning in AI: two ways to teach a model your own documents

13:00 · 21.11.2026
Source: arXiv
0

A company that wants a chatbot to answer from its own contracts, manuals or support tickets has two main ways to get there. RAG fetches the right pages before the model answers. Fine-tuning retrains the model on your examples. People mix them up, and they solve different problems.

How RAG works

RAG stands for retrieval-augmented generation. You split your documents into chunks and store them in a vector database as embeddings, numerical fingerprints of meaning. When someone asks a question, the system finds the closest chunks, puts them into the prompt and asks the model to answer from them.

The method comes from a 2020 paper by researchers at Facebook AI Research, University College London and New York University. They named the two problems it was meant to solve.

“Providing provenance for their decisions and updating their world knowledge remain open research problems.”

— Patrick Lewis and co-authors, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Quote source: arXiv, May 2020.

Those are the two things RAG still does best. You can show which page an answer came from, and you can update the knowledge by replacing a document, without touching the model. Retrieval can also fail: if the search pulls the wrong chunk, the model answers confidently from the wrong page.

Anthropic tested one fix in September 2024. It added a short description of the surrounding document to each chunk before indexing, a method it calls Contextual Retrieval, and it reports that this cut retrieval failures by 49%, or by 67% when combined with reranking, in its tests.

How fine-tuning works

Fine-tuning trains an existing model further on hundreds or thousands of your own examples and changes its weights. It is good at teaching a format, a tone or a narrow task, such as sorting tickets into your categories. It is a poor way to store facts: the model can still mix them up, you cannot see where an answer came from, and every update means another training run.

When you need neither

In the same 2024 post, Anthropic wrote that a knowledge base under 200,000 tokens, about 500 pages, can go straight into the prompt. Its models now take a million tokens, and so do GPT-6.2 and Gemini 4, which raises that ceiling. Paste the documents in first and see whether the answers are good enough.

Our take

Maks, the Intokened editor, says LLM was the hardest AI term for him to grasp. RAG is easier once you see it as a librarian working for the LLM: it finds the pages, and the model reads them. If your problem is facts that change, use retrieval. If your problem is how the answer looks, fine-tune. Our AI glossary explains embeddings and the other terms on the way.

Figures on Contextual Retrieval come from Anthropic's own tests published on 19 September 2024. Context window sizes come from vendor documentation checked on 18 November 2026.

Published: 13:00 · 21.11.2026
Maks

Author

Maks

Trading man

I've been interested in the cryptocurrency market for a long time, am a trader, and write articles and news about my experience and crypto in simple terms.

Comments (0)

No comments yet — be the first!

Join our community

No spam, unsubscribe in one click