Semantic Enrichment for RAG

Retrieval is a system, not a vector database

A production RAG system is only as reliable as the path between a user’s question and the evidence used to answer it. Embeddings matter, but they are one part of a larger design: document parsing, structural chunking, metadata discipline, hybrid retrieval, re-ranking, evaluation, and explicit failure handling.

My goal is not to make a model sound confident. It is to make the relevant context easy to recover and the resulting answer easy to verify.

Chunk around intent

Fixed-size chunks are simple, but they often split a procedure, a constraint, or a definition at the point where it becomes meaningful. I start with document structure - headings, steps, tables, and warnings - and use semantic splitting only when a dense passage contains multiple independent claims.

Small child chunks improve retrieval precision. Their parent section supplies the context needed for a complete answer. This parent-child pattern keeps search focused without forcing the model to reason over fragments.

Use hybrid retrieval and re-ranking

Semantic search understands intent and synonyms. Keyword search protects exact terms such as product identifiers, error codes, and contractual language. Combining both through reciprocal-rank fusion creates a more dependable candidate set than either method alone.

A cross-encoder re-ranker then spends more compute on the few candidates that matter. It evaluates the query and document together, which is especially useful when several chunks look similar in vector space but only one answers the operational question.

Make quality measurable

Evaluation should test the full chain: whether the right context was retrieved, whether the answer is supported by that context, and whether it is useful to the person asking. I use representative questions, inspect failure cases, and treat weak retrieval as a system-design signal rather than a prompt-writing problem.

The practical standard is simple: the system should say less when evidence is weak, cite what it used, and make human review straightforward when the cost of error is high.

What this changes

RAG becomes more than a chat interface over documents. It becomes an information system with explicit trade-offs: precision versus coverage, latency versus depth, and automation versus human judgment. That is the level at which AI can support real decisions responsibly.