RAG Advanced Patterns – Vector DB, Hybrid Search, Chunking & Reranking

RAG Advanced Patterns — Overview

The AIP-C01 exam goes far beyond basic RAG. It tests chunking strategies, embedding model selection, hybrid search (keyword + vector), reranking, metadata filtering, and vector database architecture. This post covers the advanced patterns needed for optimal retrieval quality.

Chunking Strategies — Decision Guide

Strategy	How	Best For	Bedrock Support
Fixed-size	Split at N tokens/characters with optional overlap	Simple documents, consistent size needed	✅ Built-in
Hierarchical	Parent chunks (large context) + child chunks (precise retrieval)	Long documents needing both precision and context	✅ Built-in
Semantic	Split at topic boundaries using NLP/embeddings	Varied-length documents with clear sections	✅ Built-in
Custom (Lambda)	Your own parsing logic (tables, code blocks, headers)	Structured docs (PDFs with tables, code repos)	✅ Custom transformation Lambda

Embedding Models — Selection

Model	Dimensions	Key Feature
Titan Embeddings V2	256, 512, or 1024 (configurable)	Flexible dimensions, normalize output, multilingual
Cohere Embed	1024	Input types (search_query vs search_document), compression
Custom (SageMaker)	Variable	Domain-specific fine-tuned embeddings for specialized vocabularies

Exam tip: Lower dimensions = faster search, less storage, slight quality loss. Higher dimensions = better semantic capture, more storage/compute. Choose based on accuracy vs cost tradeoff.

Vector Database Options

Service	Algorithm	Best For
OpenSearch Serverless	HNSW, IVF	Managed Bedrock KB default, full-text + vector hybrid
Aurora PostgreSQL (pgvector)	IVFFlat, HNSW	Existing PostgreSQL workload, combine relational + vector
Amazon Neptune Analytics	DiskANN	Graph + vector combined (knowledge graphs with semantic search)
Pinecone / MongoDB Atlas	Various	Third-party options supported by Bedrock KB connector

Hybrid Search & Reranking

• Hybrid Search: Combines keyword search (BM25) with vector search (semantic). Captures both exact matches AND semantic similarity. OpenSearch supports this natively.
• Reranking: After initial retrieval (top-K results), a reranker model re-scores results for relevance. Bedrock supports Cohere Rerank and Amazon Rerank. Dramatically improves precision.
• Query Expansion: Rewrite/expand user query before search. “What is S3?” → expanded to “Amazon S3 object storage buckets pricing tiers”. Improves recall.
• Query Decomposition: Break complex question into sub-queries, retrieve for each, merge results. “Compare X and Y” → retrieve for X separately and Y separately.

Metadata Filtering

• What: Filter vector search results by metadata attributes BEFORE returning to the FM
• Example: Search only documents where department=engineering AND year>=2024
• Implementation: Bedrock Knowledge Bases support metadata filters on retrieval API calls
• Use case: Multi-tenant RAG (tenant-scoped search), date-restricted search, access-control-aware retrieval

Data Maintenance & Sync

• Incremental sync: Bedrock KB supports automatic sync when S3 source changes (event-driven)
• Scheduled refresh: EventBridge scheduled rule → triggers KB sync API periodically
• Change detection: S3 event notifications → Lambda → partial re-indexing of changed documents only
• TTL/Expiration: Remove stale vectors when source documents are deleted or expired

Exam Tips

Exam	Key Points
AIP-C01	“Improve retrieval relevance” → Reranking + Hybrid search. “Long documents losing context” → Hierarchical chunking. “Keyword exact match + semantic” → Hybrid search (BM25 + vector). “Reduce storage cost for embeddings” → Lower dimension model. “Multi-tenant RAG” → Metadata filtering. “Keep knowledge base current” → Incremental sync + EventBridge schedule. “Complex query” → Query decomposition.

AWS Certification Exam Practice Questions

Question 1:

A company’s RAG system retrieves 20 chunks from the vector store, but the FM often uses irrelevant chunks, producing inaccurate answers. The vector search returns semantically similar but not necessarily relevant results. Which technique most directly improves the relevance of retrieved chunks?

1. Increase the number of retrieved chunks from 20 to 50
2. Add a reranker model (e.g., Bedrock Rerank) after retrieval to re-score and filter the top-K chunks by relevance to the specific query
3. Switch to a larger embedding model with more dimensions
4. Reduce chunk size to 100 tokens for more precise matching

Show Answer

Answer: B — Reranking is the most effective technique for improving precision after initial retrieval. The reranker model evaluates each chunk against the specific query (cross-encoder, more accurate than bi-encoder embedding similarity) and re-scores them. You then pass only the top reranked chunks to the FM. More chunks (A) adds more noise. Larger model (C) helps but doesn’t fix relevance ranking. Smaller chunks (D) may lose context.

Question 2:

A knowledge base contains technical documentation with many exact product names and version numbers. Vector search alone misses queries like “error code XYZ-123” because semantic similarity doesn’t capture exact strings. How should the retrieval be improved?

1. Fine-tune the embedding model on technical vocabulary
2. Implement hybrid search combining BM25 keyword matching with vector semantic search
3. Increase the embedding dimension for better representation
4. Add all product names to the system prompt

Show Answer

Answer: B — Hybrid search combines keyword matching (BM25 finds exact strings like “XYZ-123”) with vector search (finds semantically related content). This covers both exact match and semantic similarity use cases. OpenSearch natively supports hybrid search. Fine-tuning embeddings (A) is expensive and still may not capture arbitrary codes. Adding to system prompt (D) doesn’t scale and wastes context window.

Question 3:

A company’s knowledge base has 10,000 documents spanning 5 departments. When a user from the Engineering department queries, they should only see results from Engineering documents. How should this be implemented?

1. Create 5 separate knowledge bases, one per department
2. Tag documents with department metadata during ingestion, use metadata filters on retrieval to scope results to the user’s department
3. Add “only return engineering results” to the system prompt
4. Create separate vector indices per department in OpenSearch

Show Answer

Answer: B — Metadata filtering is the standard approach for multi-tenant/scoped RAG. Tag each document with department during ingestion. At query time, pass a filter: {"department": "engineering"}. The vector store only returns results matching the filter. This is efficient (single index, single KB) and secure. Separate KBs (A) work but don’t scale. System prompt (C) is unreliable — the FM may still retrieve/cite other departments’ content.

Question 4:

A RAG application processes long technical manuals (200+ pages each). Fixed-size chunking loses important context because a chunk might contain a paragraph that references a figure in a previous section. Which chunking strategy preserves this hierarchical context?

1. Increase chunk size to 2000 tokens to capture more context
2. Use hierarchical chunking — retrieve precise child chunks but provide parent chunk (section-level) context to the FM
3. Add overlap of 50% between chunks
4. Use sentence-level chunking for maximum precision

Show Answer

Answer: B — Hierarchical chunking creates parent chunks (large, section-level) and child chunks (small, paragraph-level). Retrieval uses child chunks for precise matching. When child chunks are retrieved, the parent chunk provides surrounding context to the FM. This gives both precision (child) and context (parent). Large chunks (A) reduce precision. 50% overlap (C) doubles storage without solving context loss. Sentence-level (D) loses even more context.

Question 5:

A company wants to minimize vector storage costs for their knowledge base while maintaining acceptable search quality. Their current embeddings use 1024 dimensions. What is the most effective approach?

1. Delete old documents from the vector store
2. Use Titan Embeddings V2 with reduced dimensions (256 or 512) — configurable at inference time
3. Compress the vector store using gzip
4. Switch from OpenSearch to a cheaper storage service

Show Answer

Answer: B — Titan Embeddings V2 supports configurable output dimensions (256, 512, 1024). Lower dimensions = less storage per vector (256 vs 1024 = 75% reduction), faster search, lower compute costs. Quality drops slightly but remains acceptable for most use cases. This is the recommended cost optimization for embeddings. Gzip (C) doesn’t work on vector indices (they need to be searchable). Deleting docs (A) removes knowledge.

Related Posts

• RAG Architecture – Bedrock Knowledge Bases (Basics)
• Agentic AI Architecture
• GenAI Architecture – Bedrock Overview
• Amazon OpenSearch

References

• Bedrock Knowledge Bases — AWS Docs
• Advanced RAG Patterns on Bedrock — AWS Blog
• Knowledge Base Chunking Strategies — AWS Docs

Frequently Asked Questions

What is the difference between hybrid search and reranking?

Hybrid search combines two retrieval methods (keyword BM25 + vector semantic) to get a broader set of relevant results. Reranking takes retrieved results and re-orders them by relevance using a cross-encoder model. Use both together: hybrid search for better recall (finding relevant documents), reranking for better precision (ordering them correctly).

How do I choose chunk size?

Smaller chunks (100-200 tokens): more precise retrieval but less context per chunk. Larger chunks (500-1000 tokens): more context but less precise matching. Start with 300-500 tokens with 10-20% overlap. Use hierarchical chunking for long documents. Test with your actual queries — measure retrieval relevance, not just similarity scores.

When should I use a custom embedding model vs Titan?

Use Titan Embeddings for general-purpose text (documentation, articles, support tickets). Use a custom/fine-tuned model when your domain has specialized vocabulary that general models don’t capture well (medical terminology, legal language, proprietary product names). Fine-tuning is expensive — only worth it when general models measurably underperform on your specific queries.