RAG (retrieval-augmented generation) means your app first searches your documents for passages related to the user's question, then sends those passages to the language model along with the question so it writes an answer based on them. The model answers from your current data instead of its general memory, which reduces hallucinations and lets it cite sources. It is usually the right choice for FAQs, policies and catalogs, while fine-tuning suits teaching a model a consistent style or format.
- what is rag in ai
- retrieval augmented generation explained
- chatbot that answers from my documents
- rag vs fine tuning
The problem RAG solves
Ask a general-purpose model, how much is a teeth cleaning at your clinic?, and it has two bad options: apologize because it doesn't know your clinic, or invent a plausible-looking number. As we covered in LLM basics, a model is great with language but knows nothing about your business, and its general knowledge stops at its training date.
The obvious fix is to paste all your documents into the instructions. That runs into context window limits, raises the cost of every message and distracts the model. Enter RAG (retrieval-augmented generation): instead of giving the model everything, you give it only the few passages relevant to each question.
Don't ask the model to remember your business. Have your app open the right file for it at the right moment.
The result is an assistant that answers from your current policies and prices, can name the document it relied on, and clearly says I don't know when it finds nothing. This is called grounding: tying an answer to a source you can verify.
How RAG works, step by step
RAG has two phases: a preparation phase that runs once (and again whenever documents change), and an answering phase that runs with every question.
Preparation (indexing)
- 1Collect documents
FAQ pages, policies, price lists, user guides and product pages. Clean out duplicates and outdated information.
- 2Split them into chunks
Each document is cut into small, coherent chunks, such as a paragraph or a question with its answer. A good chunk carries one complete idea that makes sense on its own.
- 3Turn chunks into embeddings
Each chunk passes through an embedding model that converts it into a list of numbers representing its meaning. Chunks with similar meaning get similar numbers, even when the wording differs.
- 4Store them in a vector database
Embeddings are saved with the chunk text and its source in a vector database, which supports searching by meaning, not just keywords.
Answering (every question)
- 1Embed the question
The user's question goes through the same embedding model so it lives in the same space as the chunks.
- 2Retrieve the closest matches
The app finds the few chunks closest in meaning to the question, often combined with a classic keyword search.
- 3Build the request
The app combines the system instructions, the retrieved chunks and the user's question into a single request.
- 4Generate and cite
The model writes an answer based only on the chunks and points to the source, or apologizes if the information isn't enough.
RAG, fine-tuning or just a longer prompt?
Before building, ask what the model is actually missing. Is it knowledge (facts about your business) or behavior (a style, a format, a way of classifying)? The answer picks the tool.
| Criterion | Long prompt | RAG | Fine-tuning |
|---|---|---|---|
| Best for | Small, stable information | Large or changing knowledge | Repeated style or format |
| Updating information | Edit the text directly | Update the document and re-index | Requires retraining |
| Citing sources | Awkward | Natural and easy | Usually not possible |
| Starting effort | Very low | Moderate | Higher, needs structured training data |
| How size affects cost | Grows with every message | Bounded by retrieved chunks | Training cost, then running cost |
- Prices, policies or stock change often
- You have more documents than fit in the instructions
- You need to cite sources to build trust
- You want control over what the assistant knows and doesn't
- You want a consistent writing style in your brand voice
- You need a very specific output format, repeatedly
- You have many clean examples of the desired behavior
- Good instructions and a few in-prompt examples weren't enough
An example makes the difference clear: a clothing store in Kuwait wants an assistant that writes in its playful brand voice and knows current sizes and the exchange policy. The voice is stable behavior, usually achievable with good instructions and a few examples. Sizes and policies are knowledge that changes weekly, so they belong in RAG or a direct database lookup. Fine-tuning the sizes into the model would mean retraining with every new shipment.
Worked example: a dental clinic FAQ assistant
Take a dental clinic in Riyadh with two branches that gets the same questions every day on WhatsApp and its website: hours, approximate prices, insurance, pre-visit preparation and parking. The goal is an assistant that answers these accurately, books appointments and routes any medical question to a staff member. See our clinic use cases for more ideas.
| Source | What it contains | Chunking note |
|---|---|---|
| FAQ | Opening hours, parking, languages spoken | Each question and answer is its own chunk |
| Services list | Cleaning, whitening, braces, with general descriptions | One chunk per service, including branch name |
| Insurance policy | Accepted providers and approval steps | One chunk per provider or step |
| Pre-visit instructions | Fasting, medication, arrival time | One chunk per procedure type |
- 1Define the scope
The assistant handles administrative and general information only. It never diagnoses or prescribes.
- 2Prepare the sources
Review each document with the clinic manager, remove outdated prices and add a last-updated date to every file.
- 3Keep live numbers separate
Available slots come straight from the booking system, not from a static document that may be stale.
- 4Write strict instructions
Answer only from the provided chunks, cite the source, and route medical questions to staff.
- 5Test and launch
Thirty real questions from the WhatsApp history, then a pilot on the website before WhatsApp.
Build an AI assistant for a two-branch dental clinic in Riyadh that answers in Arabic and English about FAQs, services, insurance and pre-visit instructions, using only the documents I upload, and names the document it used in every answer. If it can't find the answer or the question is diagnostic, it apologizes politely and offers to hand over to reception. Add appointment booking with phone-number sign-in, and an admin dashboard where I upload and update documents and see the questions it couldn't answer.
Try it in FanannPreparing your data: half the work happens before any code
Most RAG problems aren't in the model; they're in the documents. An assistant reading contradictory files will give contradictory answers, however strong the model is.
- One source of truth: if your return policy appears in three places with three wordings, keep one and delete the rest.
- Right-sized chunks: too small loses context, too large mixes topics. A question and answer, or a single paragraph, is a good starting point.
- Clear labels inside each chunk: include the service or branch name in the chunk itself, because it will be read apart from the rest of the document.
- Scanned PDFs: Arabic text extracted from them can come out reversed or broken. Review it before indexing.
- Bilingual content: if your documents mix Arabic and English, make sure the embedding model handles both well and test search in each.
- Metadata: store the source, update date and branch with each chunk so you can filter results and cite precisely.
Plan for updates from day one: who edits the documents, how often are they reviewed, and how are they re-indexed? The best setup is a simple dashboard where staff upload a new file and it is automatically chunked, indexed and the old version removed. That way the assistant staying truthful doesn't depend on someone remembering to email a file to a developer; it becomes part of daily operations, like updating shelf prices.
Common pitfalls and how to avoid them
| Pitfall | Symptom | Fix |
|---|---|---|
| Irrelevant chunks retrieved | Fluent answer about the wrong topic | Improve chunking, add keyword search, filter by branch or category |
| Stale documents | A price or time that's no longer true | Update dates on every file, regular reviews, delete old versions |
| Model ignores the chunks | Answers from general knowledge | Explicit instructions to answer only from chunks, plus required citations |
| No permission to decline | Invented answers when info is missing | Allow I don't know with an option to reach staff |
| Too many chunks retrieved | Slower, costlier, less focused | Retrieve only a small number of the closest chunks |
How to measure your assistant's quality
Don't stop at trying two questions and liking the result. RAG quality is measured with a fixed set of tests you rerun after every change, known as evals.
- Collect 30 to 50 real questions from your chat history, in MSA, dialect and English.
- For each one, write the correct answer and the document it should come from.
- Add deliberately out-of-scope questions to confirm the assistant declines instead of inventing.
- Score each answer: did it retrieve the right chunk, is the answer correct, did it cite the source?
- Rerun the tests after any change to documents, instructions or the model.
After launch, watch the log of questions the assistant couldn't answer. That list is gold: every repeated question points to information missing from your documents, or to a service your customers want that you hadn't noticed.
From idea to a working assistant
RAG isn't a research project; it's a clear build pattern: clean documents, thoughtful chunking, meaning-based search, strict instructions and continuous testing. With a platform like Fanann, you can describe all of it in a conversation, and the Agent builds the app, the document upload screen, the database and hosting, with full code ownership.
- Start with one task and a few trusted documents.
- Pull live numbers from the database, not from documents.
- Test with real customer questions before and after launch.
- Review unanswered questions weekly and update your documents.
Once your assistant answers accurately, the next step is letting it act, like rescheduling an appointment or tracking an order. Read about that in AI agents explained, and sharpen your instructions with examples from our prompting guide.
Key takeaways
- RAG = search your data first, then ask the model to answer based on what you found.
- Updating knowledge with RAG is easy: edit the document and re-index it, no model retraining required.
- RAG quality starts with data quality and how you split it, not with model power alone.
- Choose fine-tuning to teach style and format; choose RAG to supply facts that change.
- Test with real questions in MSA and dialect before launch, and track the questions it couldn't answer.
Frequently asked questions
Does RAG require training my own model?
How many documents do I need to start?
Does RAG eliminate hallucinations?
Does RAG work well with Arabic and dialects?
Where are my documents stored?
Ask your AI assistant about this
Copy this prompt into the AI assistant you use, then add the details of your own case.
I am reading this guide: “RAG explained: answers grounded in your own data” https://fanann.com/en/blog/rag-explained/ My question: what is rag in ai Use that page as a source and explain the answer step by step for my situation, tell me what to check before I start, and cite the page link.