Why enterprise RAG can break after the demo
This Reddit post says internal enterprise RAG systems can work in small demos but fail when connected to live business data. The author says teams often blame the model or prompt engineering, but the real problem is usually earlier in the information setup. They point to disconnected chunks from Slack, SharePoint, and Salesforce, plus old and new document versions mixed together.
Key points
- The post describes RAG systems that work on a few PDFs but fail with live enterprise data.
- The author says teams often blame the model instead of the information setup.
- Slack, SharePoint, and Salesforce content can become disconnected chunks in a vector store.
- Old and current versions of the same guide can get mixed together during retrieval.
- The post argues that retrieval quality should be fixed before swapping models.
Quick term guide
- enterprise
- A large business or company, which usually buys special software plans for better security and privacy guarantees.
- business
- An activity where you provide value to others in exchange for money.
- prompt engineering
- The practice of carefully wording instructions to get better results from an AI tool.
- AI agents
- AI agents are AI tools that can carry out steps toward a goal, not just answer once.
- AI agent
- An AI program that can inspect information and suggest what to do next.
- retrieval
- The step where a system finds the most relevant text for a question.
- scaling
- The process of making a website or app able to handle a much larger number of visitors without slowing down.
- retrieval quality
- How well a system finds the right information before answering.