Excelrs
RAG: answer from your files, not the whole internet
RAG is search plus a short brief. When to use it, how the pieces fit, and how to catch a lie.
Sandeep Jaiswal·15 March 2026 · 5 min read
RAG means the model answers from your files, not from everything it remembers. This post teaches when to use it, how the pieces fit, and how to catch a lie.
The problem RAG solves
A chat model has two bad habits with documents.
It pretends it read your PDF when you only named the title. It also stuffs twenty pages into the prompt, then loses the one paragraph that mattered.
RAG — retrieval-augmented generation — is the middle path. You keep the files. At question time you search them, pull a few chunks, and send those chunks with the question. The model writes from what you retrieved. If the chunk is missing, a good system says it does not know.
RAG is search plus a short brief. It is not “the model memorised my folder.”
The four pieces
1. Chunks. You split a PDF or a notes folder into small passages. A page is often too big. A sentence is often too small. A heading plus two or three paragraphs is a useful default.
2. An index. You store those chunks so you can search them later. The search can be keywords. It can also be embeddings — a number-vector that is close when two passages mean similar things. Beginners can start with keywords on a small folder.
3. Retrieve. A question comes in. You take the top few chunks. Three to eight is plenty. More is not kinder. More is noise.
4. Generate, with citations. The prompt says: answer only from these passages, and name the heading or page you used. Then you can open that heading and check.
If you cannot point at the chunk, you do not have RAG. You have a confident paragraph.
When not to use it
If the file is three paragraphs, paste the three paragraphs. An index is extra machinery.
If the question is “what is a for-loop?”, your internship report is the wrong shelf. Use a general model, or a lesson note on that topic.
If the answer must be an exact number from a table — a price, a date, an attendance count — prefer a tool that reads the cell, not a chunk that talks about the table.
RAG is for “answer from this set of documents.” It is not for “do a real action” (that is a tool) and not for “run five steps and recover” (that is an agent graph).
A first project
Job: a bot that answers from one of your own PDFs or a notes folder you wrote.
Done looks like this:
| Case | What should happen |
|---|---|
| A question the file can answer | The reply names a heading you can open, and the sentence is true |
| A question the file cannot answer | It says it does not know. It does not invent a section |
| A question that tries to ignore the file | It still stays inside the chunks |
Write those three cases before you build. Then you are not judging the bot by how fluent it sounds.
Keep the folder small. One report is enough. A dump of every PDF you ever downloaded will make the first search look clever and the citations look random.
How to see a lie
Ask for the heading or the page in the same reply as the answer. Then open it.
If the heading does not exist, the model invented the source. That is the bug. Do not fix it by adding “please be accurate” to the prompt. Fix the retrieve step, or refuse when no chunk scores well.
If the heading exists but the sentence is not on that page, the model used the citation as decoration. Same bug.
A short log helps: question, chunk titles, answer. When something is wrong you can see whether retrieve failed or the write-up failed.
The unknown-fact case is the one people skip. That is the case that shows whether RAG is real.
Common mistakes
One giant chunk. If each “chunk” is the whole PDF, you are back to stuffing the prompt.
No refuse path. If every question gets an answer, you trained yourself not to look.
Pretty quotes, wrong page. A citation that you never click is worse than no citation.
Mixing your files with the open web. Then you cannot tell which sentence came from the report.
Chasing a fancier index before three tests pass. Keywords on one PDF will teach you more than a vector database you do not inspect.
Practise
- Pick one file you wrote.
- Split it into chunks you can read out loud in under a minute each.
- Ask a question you already know the answer to.
- Ask a question the file cannot answer.
- Click every citation. If you cannot, the project is not done.
Checklist
- You can say RAG in one sentence: search your files, then write from the hits.
- You know when to paste the file instead.
- You have three test questions written down.
- Every answer you trust has a chunk you opened.
- The unknown-fact case says it does not know.
The next step is not a bigger index. It is one file, three questions, and a citation you actually clicked.