LLM Council upgraded: Docker, MCP, Ollama + cloud models in one open-source tool
A developer expanded Andrej Karpathy's "LLM Council" idea — where multiple AI models debate each other to reach a better answer — by adding Docker, MCP support, web search, and both local (Ollama) and cloud models in the same workflow. It's a self-hostable multi-agent setup anyone can run. For Hermes Agent users curious about chaining multiple AIs together, the architecture is worth a look.
The LLM Council concept works like a panel discussion: several AI models each answer the same question, then critique each other's responses before settling on a final answer. This new implementation wraps that idea with Docker (which runs the whole thing in an isolated environment so it doesn't interfere with your PC), MCP (a standard protocol that lets AI tools communicate), pluggable Skills, and live web search. Crucially, you can mix a locally-running Ollama model with cloud services like Claude or GPT in the same "council session," keeping costs down while still accessing more powerful models when needed. Around the same time, a Hacker News post showcased CraftBot — an agent that controls your PC and spins up real web apps on the fly using a concept called Living UI — signaling that multi-agent, tool-using AI setups are quickly becoming a practical reality across the community.
Key points
- Multiple AI models debate the same question and cross-check answers, improving quality over a single model
- Docker keeps the setup isolated — no risk of it breaking your existing tools or environment
- Mix free local models (via Ollama) with paid cloud models in one workflow to balance cost and capability
- MCP support means it can potentially connect with Hermes Agent and other MCP-compatible tools
- Web search and skill plugins let the council do real research, not just chat
Quick term guide
- LLM Council
- A setup where several AI models each answer the same question and then review each other's answers to reach a better final result
- AI models
- The core brain or underlying program that powers an artificial intelligence tool.
- self-hostable
- Software you install and run on your own server instead of using someone else's cloud service
- self-host
- To run a website, app, or service on your own server instead of using a hosted provider.
- multi-agent
- A setup where several AI agents each handle a different subtask and work together to complete a larger goal.
- Hermes Agent
- It appears to be a tool or community for building and managing AI agents.
- Cloud services
- Using powerful computers owned by other companies via the internet.
- local model
- An AI model you run directly on your own computer, with no internet connection or external service needed.
Sources covering this story (3)
- r/mcpLLM Council upgraded: Docker, MCP, Ollama + cloud models in one open-source tool ↗
- Hacker NewsShow HN: The agent that builds and operates its own SaaS tools ↗
- r/ollamaI took Andrej Karpathy's LLM Council concept to the next level (Docker, MCP, Skill, Search, local (Ollama)/cloud model support and much more) ↗