Multi-agent research system · 2026
Warrant
A five-agent LangGraph pipeline that takes a research question, plans an investigation, gathers evidence from the live web and a private corpus, fact-checks itself, and returns a fully cited report.
- Citation accuracy
- 100%
- Faithfulness
- 95.9%
- API cost vs. baseline
- −38%
- Avg Latency / report
- 46.5s
RAGAS, 16-item golden set
The problem
Research agents fail in a specific way: they produce fluent reports whose citations don't support the claims attached to them. A reader can't tell the difference without checking every source by hand, which defeats the point. The hard part isn't generating the report; it's making the citations trustworthy enough that you don't have to.
Approach
01Five agents, one graph
Planner decomposes the question, Researcher gathers evidence from Tavily web search and a pgvector corpus, Critic scores groundedness and flags gaps, Writer drafts, and a Citation Validator checks every claim against its cited span before anything is returned. State is persisted, so a run is inspectable and resumable rather than a black box.
02A critic gate that earns its cost
The Critic is the expensive node, since it re-reads evidence and can send the graph back for another research pass. Left always-on it burned latency for marginal gains, so it's gated on a threshold: it only fires when groundedness drops below 0.70 and the Critic has flagged at least 2 gaps. On the eval set that triggers on exactly 3 of 16 items.
03Evaluated as an ablation, not a vibe check
A 16-item golden dataset and a 3-arm RAGAS comparison (gate as originally written, gate off, and gate tightened), measuring hallucination rate against latency so the tradeoff was a number rather than an opinion.
Results
The tightened gate landed the best position on the curve: hallucination rate down to 4.1% from 5.5%, at 46.5s per report instead of 77.6s, roughly a 25% reduction in hallucination for close to the cost of turning the critic off entirely. Citation accuracy held at 100%, faithfulness at 95.9%, context recall at 96.9%.
| Configuration | Hallucination rate | Latency / item |
|---|---|---|
| Gate as originally written | 5.5% | 77.6s |
| Gate off | 4.8% | 40.6s |
| Gate tightened (shipped) | 4.1% | 46.5s |
What it doesn't do
- n = 16 puts the noise floor around ±1-2 percentage points, so the gap between the three arms is suggestive rather than conclusive. A larger golden set is the obvious next step.
- Faithfulness is measured by RAGAS, which is itself an LLM judge. It correlates with human judgement but doesn't substitute for it.
- Runs locally via Docker Compose; there's no hosted demo, so evaluating it means cloning the repo.
Next project
Featherweight