Phase 3 — RAG Engineering

Retrieval-augmented generation (RAG) is how you give a language model knowledge it was never trained on: your documents, your policies, your code. Instead of hoping the model remembers, you retrieve the relevant text and put it in the prompt.

RAG is the most common production AI pattern after plain chat, and it is where many AI systems succeed or fail. The model is rarely the weak link; retrieval is. This phase builds retrieval from the ground up: ingesting documents, chunking them, embedding them, searching them, ranking the results, and measuring whether any of it actually works.

What you will be able to do

By the end of this phase you should be able to:

• Explain the full RAG pipeline and where each stage fails.
• Ingest and normalise PDFs, DOCX, and HTML into clean text.
• Choose and justify a chunking strategy, and attach useful metadata.
• Explain embeddings, similarity metrics, vector indexes, and ANN trade-offs.
• Implement dense, sparse, and hybrid retrieval, and apply metadata filters.
• Improve recall with query transformation (rewriting, expansion, multi-query, HyDE).
• Rerank, compress, and select context before generation.
• Generate grounded, cited answers and version a knowledge base safely.
• Measure retrieval and generation with Recall@K, Precision@K, MRR, NDCG, faithfulness, and relevance.
• Operate RAG securely across tenants with access-controlled retrieval.

The pipeline

flowchart TD
    A["Documents<br/>PDF, DOCX, HTML"] --> B["Ingest and parse"]
    B --> C["Normalise and extract metadata"]
    C --> D["Chunk"]
    D --> E["Embed"]
    E --> F["Vector + keyword index"]

    Q["User query"] --> G["Query transformation"]
    G --> H["Retrieve<br/>dense + sparse + filters"]
    F --> H
    H --> I["Rerank"]
    I --> J["Compress and select context"]
    J --> K["Generate grounded answer with citations"]
    K --> L["Evaluate: retrieval metrics + faithfulness"]

Everything before the user’s query is offline indexing; everything from the query onward is online serving. Most RAG bugs are in one of the two, and the first job in debugging is deciding which.

Topic order

1. RAG architecture — the whole pipeline end to end.
2. Document ingestion and parsing — getting clean text out of PDFs, DOCX, and HTML.
3. Chunking strategies — fixed, recursive, semantic, and parent-child.
4. Metadata extraction and filtering — attaching and using structure.
5. Embedding models and dimensions — choosing what turns text into vectors.
6. Similarity metrics — cosine, dot product, and when they differ.
7. Vector databases and ANN indexes — HNSW and approximate search.
8. PostgreSQL pgvector — vector search in a database you already run.
9. Dense and sparse retrieval — embeddings versus BM25 and full-text search.
10. Hybrid search — combining dense and sparse results.
11. Query transformation — rewriting, expansion, multi-query, and HyDE.
12. Reranking — cross-encoders and the second stage.
13. Context construction — compression and selection.
14. Citations and grounded generation — answers you can verify.
15. Knowledge-base versioning — changing the corpus safely.
16. RAG caching and latency — making retrieval fast and affordable.
17. Retrieval metrics — Recall@K, Precision@K, MRR, NDCG.
18. Generation quality metrics — faithfulness, answer relevance, context relevance.
19. RAG evaluation and testing — offline suites, regression, and CI gates.
20. Multi-tenant RAG — isolation and shared infrastructure.
21. RAG security and access control — retrieval that respects permissions.

Tip:

How to study this phase. Keep asking “which stage is this?” Every technique here improves one of four things: the quality of the corpus, the quality of retrieval, the quality of the context, or your ability to measure the first three. If you cannot say which, the technique is not worth adding.

Checkpoint project

At the end of the phase, extend Project 1 — Production Enterprise RAG Engine: ingest PDFs and DOCX, chunk and embed them, serve hybrid search over PostgreSQL with pgvector, rerank, answer with citations, and evaluate retrieval and faithfulness on a labelled dataset. The exact scope lives in the projects part of the book.