For Excel tables, Text-to-SQL may be simpler than RAG
A Reddit user says their team has about 20 Excel files with multiple sheets. Most user questions are structured, such as counting items or checking what is required for one file. The team wants to combine the files with Python scripts and build a RAG solution with an LLM on top. The user asks whether putting the data into SQL and using Text-to-SQL would be a better setup.
Key points
- The data is about 20 Excel files with multiple sheets.
- The questions are mostly structured counts or checks against specific files.
- The team is considering Python scripts plus RAG plus an LLM.
- The poster worries RAG could add complexity, cost, and hallucination risk.
- The suggested alternative is SQL storage with Text-to-SQL for natural language questions.
Quick term guide
- Python script
- A Python script is a small program written in the Python language.
- scripts
- Small programs that run repeated tasks automatically.
- Text-to-SQL
- A method that turns a normal language question into a database query.
- AI agent
- An AI program that can inspect information and suggest what to do next.
- database
- A large collection of organized data used for search and analysis.
- guardrails
- Rules and checks that keep AI from doing harmful or unwanted things.
- guardrail
- A safety rule that prevents a system from making harmful or unwanted mistakes.
- hallucination
- When AI makes something up and presents it as a real answer.