A ready-to-use local AI system image for small makers
The idea is to make local large language models easier for regular people to use. The goal is to help people run AI on their own machines before access becomes too dependent on a few big technology companies.
The example setup uses a GMKtek Evo-X2 with 96GB of memory, either through a ready-made system image or a storage drive that is already prepared. The base system would be a slim Ubuntu server that opens into a Chromium kiosk, so the user sees a simple control screen instead of a complex setup process.
That screen would include system monitoring, open code, a friendly chat interface, a model selector, and downloaded knowledge sources such as Wikipedia and Stack Overflow through Kiwix RAG. llama.cpp would already be tuned, and an MCP server for the downloaded Wikipedia knowledge base would already be set up.
Key points
- The proposal is a ready-to-use local AI setup for non-experts.
- The example hardware is a GMKtek Evo-X2 with 96GB of memory.
- The setup would use Ubuntu server with a Chromium kiosk interface.
- It would include chat, model selection, system monitoring, and open code access.
- Kiwix RAG, llama.cpp, and an MCP server would come preconfigured.
Quick term guide
- local large language models
- AI text models that run on your own computer instead of a company’s cloud service.
- large language models
- AI models trained to read, write, and answer questions using text.
- large language model
- The type of AI behind ChatGPT or Claude — trained on huge amounts of text to read, write, and code.
- language models
- AI systems that read text and generate likely next words as answers.
- Chromium Kiosk
- A browser mode that shows one fixed app-like screen.
- Stack Overflow
- A public Q&A site where developers ask and answer coding questions.
- knowledge base
- A stored collection of information, such as documents, notes, code, or records.
- Solo developer
- An individual who handles all parts of creating a project or product alone.