Healthcare · 2026
A knowledge assistant that shows its sources
A retrieval assistant over clinical policy and procedure documents, answering staff questions with span-level citations and refusing to answer when the documents do not support one.
The problem
Policy sat across several document stores in different formats, and staff could not reliably find the current version. An assistant that answered confidently from a superseded document would have been worse than the search box it replaced.
Constraints
- Every answer had to be traceable to a span in a source document
- Results had to respect existing per-role document permissions
- Superseded documents remain in the store and must never be cited as current
- No clinical content could reach a provider without a signed agreement in place
What we built
Structure-aware ingestion with incremental re-indexing on source change, hybrid keyword and dense retrieval with a reranking pass, and generation constrained to cite the spans it used. Permissions were enforced at query time against the existing identity provider, and the system was given an explicit path to answer that it did not know.
What we built
A retrieval assistant is easy to demonstrate and hard to trust. The demonstrable part — embed the documents, retrieve the nearest chunks, generate an answer — was the smallest piece of this. The work was in the three properties that decide whether staff can rely on it: the citation is real, the permissions hold, and a superseded document is never presented as current.
Structure-aware ingestion. Policy documents have headings, tables and numbered clauses, and a fixed-size chunker destroys all three. Chunks follow document structure, so a citation points at a clause rather than at an arbitrary 800 characters.
Incremental re-indexing. Sources change on their own schedule. A changed document re-indexes on change rather than on a nightly sweep, because the window between the two is exactly when a stale answer gets given.
Hybrid retrieval, then rerank. Dense retrieval finds the paraphrase; keyword search finds the clause number someone typed verbatim. A reranking pass over the union is what makes the top few results worth generating from.
Constrained generation. The model answers from the retrieved spans and cites them. Where the spans do not support an answer, the correct output is that it does not know.
Permissions are enforced at query time
The tempting design is to filter after retrieval. It is also wrong: a document the user may not read has already influenced the ranking, and in the worst case its content reaches the model.
Permissions are resolved against the existing identity provider before retrieval, so the candidate set never contains a document the person asking could not open themselves. The assistant has no privileges of its own.
How we knew it worked
Retrieval and generation were scored separately. That distinction is the whole evaluation design: a fluent answer built from the wrong document is a retrieval failure, and a system scored only on answer quality will happily hide it.
How we evaluated it
A labelled question set drawn from real staff queries scored retrieval recall and precision separately from answer faithfulness, so a good answer from the wrong document still failed. Freshness was monitored as a production metric rather than assumed.
Results
- Answers carrying a verifiable citation
- —
- Illustrative — this is a placeholder case study.
- Retrieval recall on the labelled set
- —
- Illustrative — this is a placeholder case study.
- Median time to a cited answer
- —
- Illustrative — this is a placeholder case study.
Stack
- pgvector
- Hybrid BM25 + dense
- Rerankers — Cohere Rerank, BGE-reranker
- Embeddings — OpenAI, Voyage, Cohere, BGE, E5
- Ragas
- Python
- PostgreSQL
Tell us what you are trying to ship.
The useful first conversation is about the constraint, not the technology. Bring the deadline, the data you actually have and the thing that has to be true — we will tell you what it takes, including when the answer is that you do not need us yet.
contact@algologix.coWe reply within 24 hours.