ENTERPRISE RAG
Enterprise RAG that does not hallucinate — and stays maintainable
Most enterprise RAG pilots do not fail at retrieval. They fail three months later, when the source documents have moved, nobody owns the index, and the answers are still fluent but quietly wrong. Getting a demo to work is a weekend. Keeping it correct for two years is the actual engineering problem, and it is a maintenance problem before it is a model problem.
The five failure modes
01
Stale index. Sources changed; the index did not. The system answers confidently from a document that was superseded last quarter.
02
Untraceable answers. The response is right but nobody can show why, so no reviewer can approve it and no auditor can accept it.
03
Terminology drift. The same concept appears under three names across departments; retrieval splits the evidence and each answer sees a third of it.
04
No ownership. The pilot had an engineer; production has nobody whose job is the knowledge base. Quality decays with no alarm.
05
Chunking chosen once. The splitting strategy was tuned for the demo corpus and never revisited as document types changed.
The evidence chain, and why it is not optional
An evidence chain means every assertion in an answer is bound to a retrievable source span, and answers that cannot bind are refused rather than smoothed over. This changes what the system is allowed to do: it can say "the retrieved material does not answer this", which a fluent-by-default system never says.
Implementing it is unglamorous. Each chunk keeps its source URL, title, timestamp, and position. Retrieval returns spans, not just documents. The generation step is instructed to cite per claim and the output is validated against the retrieved set before it reaches the user. Claims without support are stripped, and if too much is stripped the answer is withheld.
The payoff is that review becomes possible. A domain expert can check ten answers by checking ten citation lists, which takes minutes. Without the chain, verifying one answer means reading the whole corpus, which means nobody verifies anything and the system is trusted on vibes.
Terminology is upstream of retrieval quality
Retrieval quality is capped by whether the corpus and the question use the same words for the same thing. In practice they do not: engineering says one term, sales says another, the regulator says a third, and the overseas buyer searches in a fourth language. Embedding models absorb some of this and miss the domain-specific rest.
The fix is a maintained glossary that is part of the system, not a document. Canonical term, accepted variants, the language mappings, and the deprecated forms — versioned, and applied at both index time and query time. This is the least fashionable component of a RAG system and consistently the one with the highest measurable effect on answer quality.
What maintainable actually costs
Budget for maintenance as a standing line item, not a contingency. Sources move and need re-crawling on a schedule tied to how fast they change, not a uniform nightly job. The glossary needs an owner and a review cadence. Answer quality needs sampling — a fixed number of production answers reviewed per period against the evidence chain, with results feeding back into the glossary and chunking strategy.
A system without these three does not stay correct; it degrades silently, which is worse than failing loudly, because the failure is discovered by a customer rather than a dashboard. If an organisation cannot commit to the maintenance, the honest recommendation is a narrower system over a smaller, slower-moving corpus — not a broader one that will rot.
Why do enterprise RAG systems hallucinate even with the right documents?
Because retrieval returning a relevant document is not the same as the answer being grounded in it — without an enforced evidence chain, the model is free to fill gaps between retrieved fragments with plausible text. The fix is structural, not a better prompt: require every claim to carry a citation, and reject answers that cannot produce one.
Where this leads
Knowledge Intelligence on PVL Nexus
The implemented version: research, acquisition, semantic indexing, evidence retrieval, governance.
What is an AI Knowledge Infrastructure Company?
Why the knowledge layer has to be modelled before anything is automated.
PVL Labs
Open-source and production evidence, including the dual-rail LLM router.
If you already have a stalled pilot
A RAG pilot that impressed everyone and then quietly stopped being used is a diagnosable condition, and usually the diagnosis is one of the five failure modes above. Tell us what it does and where it lost trust.