Fixed a dumb model by correcting its chat template
A developer discovered that a seemingly "stupid" new AI model was actually working fine once its chat template was corrected. It shows that how you structure the conversation is just as important as the model itself.
The author spent a weekend debugging the Qwen 3.6 model, which initially failed simple tasks and gave messy answers. They discovered the issue wasn't the model's brain, but the chat template—the invisible instructions that tell the AI where a user message starts and ends. Although the model was advertised as standard, it required a very specific setup hidden deep in its tokenizer_config.json file. Once adjusted, the model's performance improved significantly. This means building reliable AI agents requires strictly verifying these technical settings instead of trusting a model card.
Key points
- Model performance drops significantly if the conversation format is even slightly wrong.
- Do not trust "compatible" labels without checking the actual configuration file rules.
- Create a testing set called evals to prove a model works before fully switching to it.
- Using a middle-man tool makes it easier to swap between local and cloud AI models.
Quick term guide
- chat template
- A set of rules that formats a conversation so the AI can understand who said what.
- tokenizer_config.json
- A technical file that contains the specific rules for how an AI processes text.
- tokenizer
- The tool that splits text into tokens before the model sees it.
- AI agents
- AI agents are AI tools that can carry out steps toward a goal, not just answer once.
- AI agent
- An AI program that can inspect information and suggest what to do next.
- model card
- A document that explains what an AI model can do and how it was trained.
- cloud AI
- AI that runs on another company’s servers instead of your own computer.
- AI models
- The core brain or underlying program that powers an artificial intelligence tool.