A RAG assistant teams actually trust

A team had a sprawling knowledge base nobody could navigate. They wanted an assistant that answered questions accurately — and, crucially, that they could trust enough to put in front of customers.

The problem with most RAG

Retrieval-augmented generation is easy to demo and hard to trust. Answers look confident whether or not the sources support them, and quality silently erodes as the corpus changes. Without measurement, "it seems fine" is the only signal.

What we built

• Grounded retrieval — answers cite their sources; unsupported claims are surfaced, not hidden.
• Evals — a labelled question set scores groundedness and accuracy on every change.
• Observability — each answer logs its retrieved chunks and prompt, so any bad answer is debuggable.

Keeping it accurate over time

The corpus grows and shifts, so we run the eval suite in CI and alert on regressions. When retrieval quality dips, the team sees it on a dashboard — not in a customer complaint.

The outcome

An assistant that gives cited, grounded answers and a measurement loop that keeps it honest as the knowledge base evolves. Trust came from the tests and traces, not from a polished demo.