Healthcare domain RAG implementation case study
NDC KB Agent RAG Implementation
NDC KB Agent RAG Implementation
Standard RAG retrieves documents by semantic similarity and stuffs them into a prompt. That works for general-purpose question answering. It does not work when your knowledge base contains contradictory information from sources with different levels of institutional authority, when you need to manage a strict token budget across two different agent types, and when factual claims need to be verifiable after the fact.
This is the technical documentation of the RAG pipeline that powers the NDC KB Agent — a production system where retrieval is not just about finding relevant documents, but about finding the right documents, ranked by trustworthiness, assembled into a cache-optimized context window.
Vector Store Architecture
The vector store runs on ChromaDB with persistent storage, configured for cosine similarity using HNSW (Hierarchical Navigable Small World) indexing. The embedding model is all-MiniLM-L6-v2 — a deliberate choice that balances embedding speed (~80ms per query) against quality for a product knowledge domain where vocabulary is relatively constrained.
Each document is indexed with structured metadata:
- doc_id: Unique document identifier
- product_id: Which product this document describes
- authority_layer: catalog, marketing, or rfp
- content_type: Specs, Messaging, Sales_Play, Positioning, or RFP_Content
- market_segment: Payer, Provider, Self-Funded, or empty
- page_count: Document length indicator
This metadata is not decorative. Every field participates in retrieval filtering, authority ranking, or both. The authority_layer field drives the re-ranking system. The content_type and market_segment fields enable pre-filtering before vector search even runs, reducing the candidate set and improving both speed and relevance.
The Authority Hierarchy Problem
Enterprise knowledge bases contain contradictions. Marketing copy says "up to 40% cost reduction." The product catalog says "15-25% typical reduction." An RFP response from last quarter says "demonstrated 30% in pilot." All three documents will score high on semantic similarity for a cost reduction query. Standard RAG will pick whichever happens to have the highest cosine similarity, which may or may not be the source you want cited in a customer-facing response.
The authority hierarchy imposes institutional trust rankings:
| Layer | Authority | Role |
|---|---|---|
| Product Catalog | 100% | Source of truth — what the product actually does |
| Platform Messaging | 95% | Application of truth — how to talk about it |
| Competitive Intel | 80% | Contextual comparison — how it compares |
| RFP Templates | 70% | Contextual expansion — how it has been positioned |
These are not absolute rankings. They shift based on what the user is trying to accomplish:
Product Lookup goal: Catalog first, then Marketing, then RFP. You want facts, not positioning.
RFP Specs goal: RFP content first, then Catalog, then Marketing. You want formatted response content backed by facts.
Agent Context goal: All layers weighted equally. For internal team handoffs, breadth matters more than hierarchy.
Sales Marketing goal: Marketing first, then Catalog, then RFP. You want persuasive messaging backed by facts.
The re-ranking sort key is (authority_weight, relevance_rank) — authority is the primary criterion, semantic relevance breaks ties within the same authority layer.
The Retrieval Pipeline
Every query flows through four steps:
Step 1: Product Extraction
Before touching the vector store, the system extracts product mentions from the query text. The ElementalTagsService handles three types of matches:
- Exact: "HBA," "PHA," "VBC Analytics"
- Fuzzy: "population health" resolves to PHA, "bundled payments" resolves to BA
- Context-aware: If earlier conversation turns mentioned specific products, those are included in the extraction
Product extraction serves two purposes: it generates filter clauses for the vector search (reducing the candidate set), and it feeds the View Compiler's elemental tags section (providing the LLM with structured product context beyond what the retrieved documents contain).
Step 2: Query Formulation
In the current implementation, the user query passes through to vector search directly. The architecture includes a query formulation stage for future LLM-based query expansion — transforming "What does HBA cost?" into "HBA Healthcare Benefits Analytics pricing cost subscription license" — but the current system relies on the embedding model to handle semantic variation.
Step 3: Vector Search
The embedding model encodes the query, and ChromaDB returns candidates ranked by cosine similarity. The search uses the product extraction results as where filter clauses, so if the query mentions HBA, only HBA-related documents are candidates. The n_results parameter is set higher than the final document count to give the re-ranking step headroom — typically 2x the target retrieval count.
Step 4: Authority Re-Ranking
Each candidate document receives an authority weight based on the current user goal. Documents are then sorted by (authority_weight, relevance_rank). The highest-authority, most-relevant documents float to the top.
Hybrid Retrieval: Full vs. Metadata-Only
Token budgets are real constraints, especially when you are running two agents with different context windows. The retrieval engine splits results into two tiers:
Full documents: The top N results include complete content. These provide the LLM with enough detail to generate substantive, cited responses.
Metadata-only documents: The remaining results include only metadata (title, product, authority layer, content type). These give the LLM awareness of what else exists in the knowledge base without consuming full-content token budget.
The split is tuned per agent:
| Agent | Full Docs | Metadata Docs | Approx. Retrieval Tokens |
|---|---|---|---|
| Lookup | 2-3 | 5 | ~1,250 |
| Expert | 3-5 | 10 | ~2,500 |
This hybrid approach means the Lookup Agent can answer a simple question using 2-3 full documents while still being aware of 5 additional relevant sources it could reference or escalate to the Expert for deeper analysis.
Context Assembly and Cache Optimization
The View Compiler assembles the final context window with a deliberate structure optimized for Anthropic's prompt caching:
Stable prefix (~500 tokens, cached 90%+ of the time):
- System prompt with agent instructions
- Knowledge base schema
- Authority hierarchy definition
- Session metadata
Semi-stable middle (partial cache hits):
- Elemental product tags, sorted alphabetically for cache stability
- Knowledge graph context for referenced products
- Artifact index from previous turns
Volatile suffix (never cached):
- Conversation history
- Retrieved full documents
- Retrieved metadata documents
- Current user query
The alphabetical sorting of elemental tags is a subtle but important optimization. If a session references products HBA, PHA, and CM, those tags always appear in the order CM, HBA, PHA. Without sorting, the same three products could appear in any of six permutations, defeating the cache on every turn.
Claim Verification
The correction protocol requires the system to verify its own claims against the knowledge base. When a user disputes a factual statement, the retrieval engine runs a verification search:
- If the original source document ID is known, check that document first for the disputed claim
- If the claim is not found in the original source, run a vector search using the claim text
- For each candidate document, check for keyword overlap between the claim and the document content
- Return a verification result: confirmed (with source), or not found
This is deliberately simple. The verification does not use the LLM to interpret whether a document supports a claim — it checks for textual overlap. This avoids the circular reasoning problem where you ask an LLM to verify its own LLM-generated claim. The verification result feeds directly into the 3-branch correction protocol: confirmed claims go to Branch B (agent correct), unconfirmed claims go to Branch A (user correct).
Performance Characteristics
The retrieval pipeline is fast enough that LLM generation time dominates user-perceived latency:
| Operation | Typical Latency |
|---|---|
| Product extraction | 5-10ms |
| Embedding generation | 50-80ms |
| ChromaDB search | 20-50ms |
| Authority re-ranking | 1-2ms |
| Context assembly | 5-10ms |
| Total retrieval | 80-150ms |
Scaling characteristics are predictable: below 10K documents, the current setup works without modification. Between 10K-100K documents, sharding by product line keeps search latency stable. Above 100K, you need pre-filtering tiers and potentially a dedicated index per product class.
What This Implementation Teaches
The core insight of this RAG implementation is that retrieval quality is not just about finding relevant documents — it is about finding authoritative documents, managing token budgets deliberately, and optimizing the context window structure for caching. If you are building a production RAG system for enterprise knowledge, start with three questions: What is your authority hierarchy? What are your token budgets? And how stable is your prompt prefix? The answers shape your entire retrieval architecture.