Retrieval for chat · grounded answers
Copilots, support bots, AI assistants
Your RAG pipeline needs retrieval that keeps up with your LLM without dominating its cost. GraphANN® sits inside the loop as the citation layer. The LLM talks; GraphANN® cites.
Serve accurate, contextually rich answers across long conversations without the memory overhead that breaks most RAG deployments at scale.
01
User asks a question
Your chat orchestrator embeds the query (or asks GraphANN® to do it) and requests top-k passages with citations.
02
GraphANN® returns grounded passages
Passages come back with source metadata for citation rendering. Filters can scope the result by tenant, document set, source type, author, or any indexed metadata field.
03
LLM generates with citations
The orchestrator stitches passages into the prompt, calls your LLM, and renders citations back to source. GraphANN® never touches the LLM directly.
Best fit when
- You're already paying LLM latency, retrieval just needs to keep up
- You expect to swap embedding models or chat providers over time
- You need source-grounded citations, not vibes
- You want the retrieval store to be cheap to host alongside the LLM
Wrong choice when
- You only need keyword search and a regex (no semantic recall benefit)
- Your bot needs full conversational memory. That's a session store, not a vector DB
- You want a built-in cross-encoder reranker. GraphANN® returns passages and you bring your own reranker on top.
~50ms
median query time
<5%
of LLM turn latency (retrieval share)
Any
LLM provider