AI APIs charge for 'thinking' tokens you never see in the response
When you use certain AI models, you're billed not just for the text they send back, but also for an internal reasoning process that stays hidden. This can make your actual costs much higher than what the visible response suggests.
'Reasoning models' like OpenAI's o1 and o3, or Claude with extended thinking enabled, work through a long internal thought process before producing an answer. Every token used in that hidden reasoning counts toward your bill, even though you never see it. For example, if a response looks like 200 tokens but the model spent 2,000 tokens thinking first, you're charged for 2,200 tokens total.
This catches many developers off guard and can blow up a budget quickly. To keep costs under control, always check the usage field in the API response to see the true token count, set a budget tokens limit if the model supports it, and use a standard (non-reasoning) model for tasks that don't need deep reasoning.
Key points
- Reasoning models (o1, o3, Claude extended thinking) consume tokens internally while 'thinking' — those tokens never appear in the reply but are still billed
- Your actual API cost can be many times higher than the length of the visible response alone
- Always inspect the usage field in the API response to see the real total token count
- Set a budget tokens cap where supported to put a hard ceiling on reasoning token spend
- For straightforward tasks, use a standard model instead of a reasoning model to avoid paying for unnecessary hidden thinking
Quick term guide
- AI models
- The core brain or underlying program that powers an artificial intelligence tool.
- AI model
- A program that can understand prompts and produce text, code, or answers.
- AI Mode
- A Google Search feature that uses AI to answer longer, more detailed questions.
- reasoning
- The ability of the AI to think through complex steps to find a solution.
- extended thinking
- A mode where an AI works through a detailed internal reasoning chain before giving its answer, using extra tokens in the process.
- developers
- Developers are people who build software, apps, or websites.
- usage field
- A section of the API response that reports exactly how many input, output, and reasoning tokens were consumed.
- budget tokens
- A setting that caps how many tokens a reasoning model is allowed to spend on its internal thinking, limiting surprise costs.