Catalog-wide RAG questions need a different path than normal search
An enterprise product document assistant works reasonably well for narrow questions about one product, comparisons between two products, summaries of uploaded documents, and answers with citations. It struggles with catalog-wide questions such as which products mention a term, support a feature, have a certification, miss an attribute, or contain a specific model term.
The answer can look reasonable and include citations, but still leave out matching products. The main issue is not hallucination; it is incomplete coverage because the language model only sees the retrieved document chunks.
Normal vector RAG returns a small set of likely relevant chunks, while the user expects every active product to be checked. A more reliable design is to detect catalog-wide questions and route them to a deterministic catalog coverage path that checks all active products.
Key points
- The assistant handles narrow product questions and document summaries better than catalog-wide checks.
- Questions across 50-80 products can miss valid matches even when the answer looks credible.
- The failure comes from incomplete retrieved chunks, not mainly hallucination.
- Normal vector RAG is a poor fit when every active product must be checked.
- A deterministic catalog coverage path can handle these questions more reliably.
Quick term guide
- enterprise
- A large business or company, which usually buys special software plans for better security and privacy guarantees.
- hallucination
- When AI makes something up and presents it as a real answer.
- language model
- An AI model that reads and writes human language.
- vector RAG
- A RAG method that turns text meaning into numbers to find similar content.
- deterministic
- Giving the same result every time when the input is the same.
- reliability
- How consistently a tool works without failing or behaving unexpectedly.
- AI agents
- AI agents are AI tools that can carry out steps toward a goal, not just answer once.
- token cost
- The money or usage spent when sending text to an AI model and getting text back.