AI Daddy › Retrieval Systems
Advanced Retrieval Patterns · Retrieval Systems
Beyond the basics, production RAG systems use specialized patterns to handle complex query-document gaps. These patterns are the "secret sauce" of…
Advanced Retrieval Patterns
Beyond the basics, production RAG systems use specialized patterns to handle complex query-document gaps. These patterns are the "secret sauce" of high-precision search and are increasingly bundled into managed RAG offerings.
Table of Contents
Query Decomposition (Multi-Query)
Complex user queries are often "Compound Queries."
- User: "Compare our Q3 vs Q4 revenue and explain the drop."
- Decomposition:
- "Q3 Revenue"
- "Q4 Revenue"
- "Reasons for Q4 revenue variance"
- Implementation: Use an LLM to generate these 3 sub-queries, search the DB for ALL of them, and aggregate the context.
Hypothetical Document Embeddings (HyDE)
Queries are short; documents are long. This "Asymmetry" causes retrieval failure.
- Pattern:
- Take the user query.
- Ask the LLM: "Write a 1-paragraph hypothetical answer to this."
- Embed the hypothetical answer instead of the query.
- Why?: The hypothetical answer is in the same "Vector neighborhood" as the real documents, leading to much higher recall.
Contextual Retrieval (The Anthropic Pattern)
standardized by Anthropic in late 2024, this pattern solves Context Dilution.
- The Problem: A chunk might say "It costs $200," but without the header, we don't know "It" is a "Widget-X."
- The Pattern: During ingestion, for every 300-token chunk, have an LLM write a 50-token context string (e.g., "This chunk is about the pricing for Widget-X in the North American market").
- Benefit: Increases retrieval precision by 30-50% for fragmented data.
Iterative Document Enrichment
Instead of just storing the raw document, we store "Enriched" meta-data.
- Summary: Store a 1-paragraph summary of the document.
- Q&A Generation: Generate 5 questions this document answers and embed those with the document.
- Status: Most high-end RAG systems now embed "Questions" rather than "Answers" to match the user's query intent.
In-Context Reranking
With 1M-2M context windows now standard (Claude Sonnet 4.6, Gemini 3.1 Pro), Rank-by-Context is a viable pattern.
- Retrieve Top 100 docs.
- Put all 100 in the context window.
- Ask the model: "Read these 100 docs and identify the 5 most relevant. Then, use those 5 to answer."
- Win: This utilizes the model's Long Context Reasoning to perform reranking without needing a separate Cross-Encoder model.
Interview Questions
Q: Why is HyDE (Hypothetical Document Embedding) risky for some applications?
Strong answer:
HyDE relies on "Hallucinating" a baseline answer to find real data. If the user's query describes something non-existent or logically impossible, the LLM will still generate a hypothetical answer. This can pull in "Incorrect but Semantically Similar" data from the database, reinforcing the model's initial hallucination. The standard mitigation is a Hybrid approach: retrieve once with the real query (Keyword) and once with the HyDE query, then use RRF to combine them.
Q: What is the "Asymmetric Retrieval" problem?
Strong answer:
Asymmetric retrieval refers to the fact that user queries are usually short (3-10 words) while document chunks are long (300-500 words). These inhabit different statistical distributions in the vector space, leading to "Distance Bias." High-performance systems solve this using Asymmetric Encoders (one model for queries, one for docs) or Query Expansion (HyDE) to "inflate" the query into a document-like distribution.
References
- Gao et al. "Precise Zero-Shot Dense Retrieval without Relevance Labels" (HyDE, 2023/2024)
- Anthropic. "The Contextual Retrieval Playbook" (2024)
- LlamaIndex. "Query Transformation Cookbook" (2025)
Next: Agentic Systems