Book a free call

RAG vs. fine-tuning: when to use each

What retrieval-augmented generation and fine-tuning each change, what they cost, and why most production systems combine them.

Short answer

Use RAG when answers depend on facts that change or must be cited, such as policies, manuals or records. Use fine-tuning when you need consistent format, tone or domain behaviour, or a cheaper model. Most production systems use RAG first and fine-tune later if needed.

What RAG changes

RAG adds a search step before the model answers. The system finds relevant passages in your documents and asks the model to answer using only those passages. Knowledge stays outside the model, so updating a document updates the answer.

What fine-tuning changes

Fine-tuning trains the model further on your examples. It changes how the model behaves: format, style, terminology and the kinds of tasks it handles well. It is poor at storing facts that change.

Comparison

QuestionRAGFine-tuning
Keeps answers currentYesNo
Shows sourcesYesNo
Fixes format and tonePartlyYes
Can cut cost per requestPartlyYes, with a smaller model
Typical first build$50k–$150k$45k–$120k

When to combine them

A common pattern: RAG supplies the facts, and a fine-tuned small model writes the answer in your format at lower cost. Another: fine-tune a model to write better search queries for RAG.

How to decide in practice

  1. Build a test set of 100–300 real questions with good answers.
  2. Try a strong prompt alone.
  3. Add RAG and measure.
  4. Fine-tune only if a clear gap remains.

We run this comparison in our fine-tuning fit test.

Key takeaways

  • RAG is for facts; fine-tuning is for behaviour.
  • Measure on your own test set before choosing.
  • Combining both is common in production.

FAQ

Is RAG cheaper than fine-tuning?

To start, usually yes. At very high volume, a fine-tuned small model can be cheaper per request.

Can fine-tuning reduce hallucinations?

It can improve format and domain behaviour, but grounding with RAG does more for factual accuracy.

How much data does fine-tuning need?

Often a few hundred to a few thousand high-quality examples.

Want help with this?

Want us to test RAG and fine-tuning on your data? Book a free call or see our rag development service.

Estimate costBook a call