Retrieval-augmented generation (RAG)
Retrieval-augmented generation is a method that looks up relevant material from your approved documents before an AI writes an answer, so replies are grounded in your files rather than the model's training data.
A large language model answers from patterns it learned during training. That works for general knowledge. It fails on your contracts, policies, prices or procedures, because those files were never in the training data. Retrieval-augmented generation (RAG) searches a store of your approved documents first, pulls the matching passages, then asks the model to write an answer from them.
In practice the store is a document library or indexed files from systems such as SharePoint. A member of staff types a question. The system retrieves the closest extracts, then the model drafts a reply that can point back to the source file. Operations teams use this for policy queries, contract clauses, product specifications and first-line helpdesk answers. You decide which libraries are in scope and who can query them. The ongoing work is keeping those sources current.
RAG goes wrong when retrieval fails. Duplicate files, outdated versions, vague names and folders that were never indexed send the model the wrong extract. It then writes a fluent answer from that extract. Access control is another failure: if the index includes material an employee should not see, the system can surface it in a reply. RAG reduces invented answers. It doesn't remove them, and it cannot supply a fact that is not in the source. Someone still has to name the authoritative documents and review outputs that affect customers or compliance.
When it matters
- →Staff keep asking the same questions of policy documents and handbooks.
- →Helpdesk answers need to cite a specific source file.
- →Product or pricing answers must come from your live documents.
- →You plan to launch an internal assistant on company files.
Related terms
Retrieval-augmented generation (RAG): common questions
Does RAG stop AI hallucinations?
Hallucination means the model invents a fact. RAG reduces that, because the model writes from retrieved passages. It doesn't eliminate it. A wrong or stale extract still produces a wrong answer, and it will often sound confident. RAG also cannot answer a question that is not covered in the source files. Treat it as a way to keep replies tied to your documents, then review anything that affects a customer or a compliance decision.
What is the difference between RAG and fine-tuning?
Fine-tuning retrains the model on extra examples so its behaviour changes. RAG doesn't retrain anything. It fetches relevant documents at the time of each question. Fine-tuning is slower to update when a policy or price list changes. RAG updates when you add, replace or remove a file in the store. For operational questions about current documents, RAG is the usual starting point. Fine-tuning is a later decision, and for a narrower set of tasks.
Is RAG the same as a chatbot?
No. RAG is a retrieval method, not a product staff log into. A chatbot is the screen they type into. You can run a chatbot with no retrieval, so it answers from general training and won't know your policies. You can also use RAG without a chat window, for example to draft a helpdesk reply from the knowledge base. The business decisions are which documents the system may search and who reviews output before it reaches a customer.
Ready to talk?
Book a free, no-obligation discovery call. We'll learn about your business and show you exactly how Wanzo can help — with a bespoke proposal within 48 hours.
No commitment. No sales pressure. Just honest advice.