Retrieval-Augmented Generation (RAG)

3.1 How RAG Delivers External Knowledge to AI

AI training data becomes outdated. RAG solves that by teaching AI to read your documents instead of relying on training data alone. Imagine a library where AI normally can only use its pre-trained knowledge (like a student with a fixed textbook). RAG is the librarian that hands AI a curated shelf of documents it needs—your company's policies, industry research, legal precedents, customer FAQs—so it answers from your authoritative sources instead of generic training data.

How RAG works:

1. Ingestion: You upload documents (PDFs, Word files, web pages, databases).
2. Indexing: The system converts documents into searchable form, creating a searchable "catalogue."
3. Retrieval: When you ask a question, the system finds the most relevant documents from your collection.
4. Context insertion: Those retrieved documents are passed to AI along with your question.
5. Generation: AI answers using your documents, not training data alone.

Why it matters: A legal research AI without RAG relies on its training data, which is months or years old. A RAG-enabled legal AI queries your Westlaw database, pulling the latest cases and statutes. The difference between stale and current is enormous.

RAG Analogy: Without RAG, AI is like a generalist who read all the business books years ago. With RAG, AI is like a specialist with a current library of domain-specific materials. (See Module 2.2 for how context engineering principles apply to RAG document selection.)

3.2 Building and Managing Knowledge Bases

Building a knowledge base requires discipline. It's not "dump all documents and hope." It's curating, organizing, and maintaining a collection that AI can search effectively.

1. Decide scope: What documents will AI read? (Your vendor contracts? Customer feedback? Industry regulations? Your product documentation?) Too broad and retrieval becomes unreliable. Too narrow and AI lacks context.
2. Audit and clean: Review documents for quality. Remove duplicates, outdated versions, and corrupted files. Consistent formatting helps AI parse them correctly.
3. Organize hierarchically: Group documents by category (legal contracts, compliance docs, technical standards, FAQ) so AI can search within relevant buckets rather than all at once.
4. Index (let platform handle this): Most RAG platforms automatically index your documents. Understand your platform's indexing method—some use keyword search, others use semantic similarity. Know the difference.
5. Create retrieval rules: Some documents are more authoritative (your official policy > general article about the topic). Assign weights or priority levels.
6. Test and iterate: Ask sample questions. Verify the system retrieves the right documents. Adjust categories and weights if retrieval is missing key materials.
7. Maintain continuously: Documents age. Update the knowledge base quarterly—remove obsolete materials, add new ones, refresh stale content.
8. Monitor retrieval quality: Track which documents get retrieved for which queries. If irrelevant docs appear, reweight or recategorize.