Shipping agentic RAG without the hype
Most RAG demos fall over in production. Here is the small set of things that actually made mine reliable — and measurable.
Retrieval-augmented generation demos beautifully and breaks quietly. The gap between a notebook and something you’d put in front of real users is mostly unglamorous engineering — and a few decisions that pay off again and again.
Make it a graph, not a pipeline
A linear “retrieve → stuff → answer” chain can’t recover when retrieval misses. Modeling the flow as a small state machine — decide whether to retrieve, retrieve, check if it’s enough, maybe retrieve again, then answer — turns dead ends into branches. LangGraph makes those branches explicit instead of buried in prompt spaghetti.
Trace everything from day one
You cannot debug what you cannot see. Wiring tracing in before the first real query means every run shows the retrieved chunks, the tool calls, and the final prompt. Half of “the model is wrong” turns out to be “retrieval gave it the wrong context.”
Evaluate on your data, not vibes
A tiny evaluation set built from real questions is worth more than any benchmark. Once each change produces a number, “this feels better” becomes “recall went from 0.71 to 0.83,” and you can move fast without breaking quality.
None of this is exotic. It’s just the difference between a demo and a system.