paddock helps pick local LLMs that fit an Apple Silicon Mac
A Reddit user announced paddock, a Rust tool for choosing local LLMs on Apple Silicon Macs. The tool reads the Mac’s chip, RAM, and Metal memory limit to estimate which models fit. It compares models from Ollama, Hugging Face GGUF, and mlx-community, and can produce run commands or an OpenAI-compatible endpoint.
Key points
- paddock checks an Apple Silicon Mac’s chip and RAM before suggesting local LLMs.
- It uses the Metal memory limit to estimate what can actually fit on the machine.
- It compares Ollama, Hugging Face GGUF, and mlx-community models.
- paddock run and paddock serve can create the needed command or an OpenAI-compatible endpoint.
- The author says it can read GGUF model details without downloading the full model file.
Quick term guide
- local LLMs
- AI language models you run on your own computer instead of through a company's online service
- local LLM
- An AI language model that runs on your own computer instead of on a remote server.
- Apple Silicon
- Apple's own line of chips (M1, M2, M3, M4, M5) used in Macs, known for performance and efficiency.
- Hugging Face
- An online place where AI models and datasets are shared.
- commands
- Instructions given to a computer or tool to do a specific task.
- compatible
- Parts that can work together without causing a mismatch.
- endpoint
- A web address where a service sends or receives data.
- Mac mini
- A small desktop computer made by Apple.