Excelrs
Prompt, then RAG, then fine-tune
Shape is a prompt. Facts live in files. Fine-tune is last, and only after you can count clean pairs.
Sandeep Jaiswal·8 December 2025 · 5 min read
When a model is wrong, people jump to fine-tune. That is the expensive fork. Most of the time you need a better prompt, or you need the file in the answer. Fine-tune is last.
This post is a simple order: prompt, then RAG, then fine-tune — and how to know you are on the right step.
The three tools
Prompt. You change the instruction or add two examples. No new files. No training job. Use this when the shape is wrong: too long, no flag, wrong tone.
RAG. You fetch chunks and ask the model to use them. Use this when the fact lives in a file that changes: a rule, a fee, a date.
Fine-tune. You train a model on many input-output pairs. Use this when the style or a narrow format is stable, and you have hundreds of clean pairs — and prompt plus RAG still fail the same way.
If the fact can change next week, do not bake it into a fine-tune. Put it in a file.
If you cannot name the fact that is missing, you are not ready for RAG or a fine-tune. Fix the prompt first.
A table you can keep
| Symptom | First try | Not yet |
|---|---|---|
| Essay instead of three bullets | Prompt + schema | Fine-tune |
| Invents a date the file has | Cite check | Fine-tune |
| File not in the window | RAG | Fine-tune |
| Same style fail on 200 clean pairs, files already in | Then consider fine-tune | More adjectives |
| Fee changed this morning | Update the file | Retrain |
Most student tools stop at row two or three.
Walk through once
You want “did I hit 75%?” from a notes file.
If the answer is a paragraph, change the prompt. Add a true example and a false example. Run five cases.
If it still invents a percentage, you do not have a style problem. You have a grounding problem. Retrieve the rule. Cite it. Stop on empty.
If the rule is in the chunk and the cite opens, you do not need a fine-tune.
A year later, if you have 300 labelled answers in the same shape, and a small model still drifts on tone after the schema, then a fine-tune can lock the shape. The rule still lives in the file. The fine-tune does not store next semester’s date.
What good looks like
You can say which door you are in this week.
If the answer is the wrong shape, you are in prompt. You have two examples and a schema. You do not have a training job.
If the answer is a fact that lives in a file, you are in RAG. You have chunks, cites, and a refuse.
If someone asks for a fine-tune, you can show the five cases and the pair count. If the pair count is twenty, you show them the few-shot post instead.
The fee that changed this morning is a file edit. It is not a reason to open a notebook.
The leftover fail has to be the same fail
Fine-tune is for a leftover that stays after prompt and RAG: the same format miss, on many clean pairs.
If Monday’s leftover is a fake date, Tuesday’s is tone, and Wednesday’s is an empty retrieve that drafted, you do not have one leftover. You have three bugs. Training will smear them together.
Write the leftover in one sentence. If you cannot, you are not ready to collect pairs.
Collecting pairs is work. Do not start that pile to avoid a cite checker. The checker is cheaper.
Walk the first two doors. The third door has a lock called “hundreds of clean pairs.”
When people fine-tune too soon
They have twenty examples. That is a few-shot, not a train set.
They want the model to “know the college.” Knowledge that changes belongs in retrieval.
They hope training will remove hallucinations. It will not replace a cite check.
They skip eval. After the job, they still cannot say which of five cases moved.
Cost of the wrong fork
A prompt change is an afternoon.
RAG is a week of chunking and cites if you do it properly.
A fine-tune is data work, money, and a model you must replace when the format changes. If you also stored facts in it, you retrain when a circular updates.
Start cheap. Keep the five cases. Only pay more when those cases name the same leftover fail.
Common mistakes
Fine-tune as a personality. You wanted a prompt.
RAG without a refuse. You wanted a search box that lies less, and you got a search box that still lies.
Mixing all three in one weekend. You will not know what helped.
Training on chat logs with jokes and passwords. You copied the mess.
No owner for the file. RAG dies the week the PDF is stale.
Practise
- Write the symptom in one line.
- Change only the prompt. Rerun five cases.
- If a fact is missing, add RAG and cites. Rerun.
- If you still think fine-tune, count clean pairs. If under a hundred, stop.
- Keep the table next to the repo.
Blank on the last box is a good week, not a gap.
Checklist
- You can name prompt vs RAG vs fine-tune in one line each.
- Facts that change live in files.
- You do not fine-tune to stop a hallucination.
- Five cases run after each fork.
- You have counted pairs before you train.
The next step is the first door, not a training form.