A practical RAG roadmap before building agentic AI
A learning-focused RAG system already includes several core parts. It can create embeddings with sentence transformers, split documents into chunks with LangChain, store vectors in ChromaDB, ingest documents with Docling, show citations and sources, rewrite queries, and use a reranker.
It does not have a frontend yet. Possible next additions include query expansion and hybrid search, but the best next step is unclear.
The long-term goal is to turn the system into agentic AI, not just a search-and-answer tool. The main question is whether too many features are being added before the system has a clear direction.
Key points
- The system already has embeddings, chunking, vector storage, document ingestion, citations, query rewriting, and reranking.
- There is no frontend yet.
- Query expansion and hybrid search are being considered as next features.
- The final goal is to move from RAG toward agentic AI.
- A clear evaluation step should come before adding more features.
Quick term guide
- embeddings
- A way of converting text into numbers so that similar meanings can be found and compared mathematically.
- Transformer
- A common AI model design that helps software understand relationships between words in text.
- citations
- Citations show which source a claim or answer came from.
- hybrid search
- A search method that combines keyword matching with meaning-based matching.
- agentic AI
- AI that tries to complete a goal by taking several steps, not just answering one question.
- AI agents
- AI agents are AI tools that can carry out steps toward a goal, not just answer once.
- query rewriting
- Rephrasing the original question so the search engine finds more relevant results.
- evaluation
- A process of testing and scoring how well an AI performed its specific task.