Large language models know a lot about the world and nothing about your business. Ask one about your refund policy, your price list or last month's invoice and it will either admit it does not know or, worse, make up a confident answer. Retrieval-augmented generation, or RAG, fixes this by giving the model the right pages from your own documents at the moment it answers. It is the most common way businesses put AI on top of their own data, and it is easier to build than most people expect.
The idea in one minute
RAG has two parts. First, a retrieval step finds the passages in your documents that are most relevant to the question. Second, a generation step hands those passages to the model along with the question and asks it to answer using only that material. The model is not retrained. It simply reads the right context before it replies, the way a support agent opens the policy page before answering a customer.
The pipeline, step by step
- Collect documents. FAQs, product sheets, policies, past support answers, manuals and help articles. Remove outdated and duplicate files first, because the system can only be as accurate as its sources.
- Split into chunks. Cut each document into passages of roughly 200 to 500 words, ideally along headings, so each chunk holds one complete idea.
- Create embeddings. An embedding model turns each chunk into a list of numbers that captures its meaning. Similar meanings end up close together.
- Store them. Keep the vectors in a vector database or in PostgreSQL with the pgvector extension, together with the original text and its source.
- Retrieve. Embed the user's question the same way and fetch the closest chunks, often the top three to eight.
- Generate. Send the question and those chunks to the model with clear instructions: answer from the context, say when it is not there and name the source.
What makes RAG good or bad
- Chunking: chunks that cut a table or a sentence in half produce half-answers. Keep related facts together.
- Metadata: store the document title, date and category so you can filter, for example by product or by language.
- Hybrid search: combine meaning-based search with keyword search. Exact terms such as product codes and names are often missed by embeddings alone.
- Re-ranking: a second model that re-orders the top results usually raises accuracy noticeably for little cost.
- Freshness: re-index when documents change, and drop old versions. A bot quoting last year's price is worse than no bot.
Common failure modes
- The answer is not in your documents, and the model guesses anyway. Instruct it to say "I do not know" and hand over to a person.
- The right chunk exists but retrieval missed it. Test with real questions and look at what was fetched, not only the final answer.
- Conflicting sources. Prefer newer documents and show the source so a human can check.
- Sensitive data leaks across users. Filter retrieval by the user's permissions before anything reaches the model.
Where RAG fits in a business
RAG suits any case where the answer lives in text you already have: customer support, internal policy questions, product guidance for sales, onboarding and proposal drafting. It is a weaker fit for exact calculations or live numbers, which should come from your database or an API. A good design lets the assistant do both: search the documents for policy and call a function for the live balance.
How we use this with WhatsApp and business software
A WhatsApp assistant is the natural front door for RAG, because customers already ask questions there. Our WhatsApp automation reads incoming messages and turns them into structured data. Add retrieval and the same channel can answer product and policy questions from your own documents, then pass the leads to your team. For daily operations, the same approach can sit inside business software such as our Lena Dena platform, where staff could ask how a tax rule applies, how to record a supplier return or what a role is allowed to do, and get an answer taken from the help content instead of searching manuals. Anything that touches money or customer data still goes to a person for confirmation.
A sensible first project
- Pick one narrow area, such as your top 50 customer questions.
- Index the documents that answer them and write 30 test questions with the correct answers.
- Measure how often the right passage is retrieved before you tune the prompt.
- Launch to your own team first, collect the failures and fix the sources.
Quick questions answered
- Is RAG the same as fine-tuning? No. Fine-tuning changes the model's behavior and style. RAG gives it fresh facts at answer time, which is easier to update and audit.
- How much data do we need? Less than you think. A few dozen good documents covering your real questions beat thousands of messy ones.
- Can it work in Hindi or Hinglish? Yes, with multilingual embeddings and a model that handles the language. Test with real messages from your customers.
If you want an AI assistant built on your own documents, see our AI and chatbot services or tell us what your customers ask most.