Every AI interaction costs tokens. Every token carries a price. And in the agentic era, a single user request can fan out into 5–50 LLM calls. This guide breaks down 2026 token economics — and gives you the exact levers to cut spend by 70–85% without sacrificing quality.
Tokens are the atomic unit of AI computation — not characters, not words, but sub-word fragments that every LLM uses to read input and write output. Understanding them is understanding the bill.
Common words are 1 token; rare words split into multiple. Code is densest. And as of 2026, tokenisers differ materially by model: Claude Opus 4.7+, Opus 5/5.5, Fable 5/5.1 and Sonnet 5/5.5 produce ~1.0–1.35× (typically ~30%) more tokens for the same text than Opus 4.6 / Sonnet 4.6 / Haiku 4.5 — so $/MTok is not comparable across tokeniser generations.
GPT-6 Astra: $10 input vs $50 output per MTok. Claude Opus 5.5: $4 vs $20. GPT-6.1 Sol and Claude Sonnet 5.5: $2 vs $10. Output is 5× input on most current flagships (Grok 4.7 and Qwen3.8-Max: 3×) — a model that writes a lot costs far more than one that reads a lot, though the 2026 price war has compressed the ratio from the old 4–10×.
GPT-6 Astra / 6.1 Sol (reasoning effort), Claude Opus 5.5 (thinking always on, effort default medium) and Claude Fable 5.1 (always on) generate internal chain-of-thought tokens that are billed as output but never shown to you — often 3–10× the visible output.
One user request can trigger 5–50 LLM + MCP tool calls. Simple agents use 5K–15K tokens per task; complex multi-agent systems consume 200K–1M+ tokens per task. This is the dominant cost story of 2026.
Your instructions to the model. 200–2,000 tokens. Sent on every single call — prime candidate for prompt caching.
All prior messages in the conversation. Grows every turn. The silent multiplier — often the largest input cost driver.
The actual query plus any retrieved documents (RAG). Usually the smallest component but grows with retrieval.
What the model writes back. Billed at typically 3–5× the input rate on current flagships. Longer responses = proportionally higher cost. The most expensive component per token.
USD per million tokens (MTok) — October 4, 2026. September rewrote the price list. Near-frontier pricing converged at about $2/$10 (GPT-6.1 Sol, Claude Sonnet 5.5 and Gemini 4 Argon's intro rate), while the very top went up and got gated: GPT-6 Astra launched at $10/$50 with a 6× 'Ultrafast' tier, and Argon is restricted to cyber-defence partners. Anthropic cut Claude Opus 5.5 to $4/$20 — and the new #1 on the Artificial Analysis index is now the cheaper Opus. The real battleground moved to cached input: Fable 5.1 reads cache at 0.025× and Opus 5.5 at 0.05×, GPT-6.1 Sol at 95% off. Meanwhile 'cheap' got more complicated: DeepSeek now bills 2× at peak hours, Google's $0.75/$3.75 Gemini 3.8 Flash price expires Dec 31, and GPT-6 and Grok reprice the entire request once a prompt crosses 272K / 200K tokens. Output still costs 3–5× input — always model the ratio for your workload.
⚠️ Tokeniser caveat: $/MTok is not directly comparable across models. Claude Opus 4.7+, Opus 5/5.5, Fable 5/5.1 and Sonnet 5/5.5 use a tokeniser that produces ~1.0–1.35× (typically ~30%) more tokens than earlier Claude models. And token appetite now varies more than price: Grok 4.7 uses ~81K output tokens per Artificial Analysis task vs ~36K for Grok 4.6 (same $2/$6 sticker, ~2× the bill); Gemini 3.8 Flash used 120M output tokens vs 64M for 3.7 Flash on the same suite (+70% cost); Sonnet 5.5 at max effort burns ~193K output tokens per task ($7.60) vs $0.59 at medium. Benchmark cost per completed task, not price per million tokens.
| Model | Provider | Tier | Input $/MTok | Output $/MTok | Cached Input | Context | Best For |
|---|---|---|---|---|---|---|---|
| GPT-6 Astra | OpenAI | Flagship | $10.00 | $50.00 | $1.00 | 1M | OpenAI flagship (Sep 3). >272K input reprices the whole request (2× in / 1.5× out). Ultrafast tier is 6× ($60/$300) |
| Claude Fable 5.1 | Anthropic | Flagship | $10.00 | $50.00 | $0.25 | 1M | Most capable Claude (Sep 1). Cache reads cut 75% vs Fable 5 (0.025×). 30-day retention, no ZDR |
| Claude Opus 5.5 | Anthropic | Flagship | $4.00 | $20.00 | $0.20 | 1M | Anthropic's default (Sep 22) — 20% cheaper than Opus 5 and #1 on the AA index (58). Cache reads 0.05×; fast mode $8/$40 |
| Kimi K3 | Moonshot | Flagship | $3.00 | $15.00 | $0.30 | 1M | Open weights (modified MIT). Flat pricing across 1M — but always reasons (~39 tok/s), so budget CoT output |
| Grok 4.7 | SpaceXAI | Flagship | $2.00 | $6.00 | $0.50 | 500K | Sep 21. Prompts ≥200K bill the whole request at $4/$12; no batch discount; ~2× Grok 4.6's tokens per task |
| Qwen3.8-Max | Alibaba | Flagship | $2.00 | $6.00 | $0.25 | 1M | Among the top Chinese models on the AA index (45, one behind MiMo-V2.6-Pro). Slow; xhigh reasoning by default inflates output |
| Gemini 4 Argon | Flagship | $2.00* | $10.00* | $0.10 | 1M | *Intro price, rising to $4/$20. Restricted to Fairwind cyber-defence partners (Sep 30); no public API yet | |
| Gemini 3.1 Pro | Flagship | $2.00 | $12.00 | $0.20 | 1M | Still preview. Classic long-context surcharge: $4/$18 above 200K input. New work → Gemini 3.8 Flash | |
| Claude Opus 5.5 (thinking always on) | Anthropic | Reasoning | $4.00 | $20.00 | $0.20 | 1M | Thinking can't be disabled; effort defaults to medium — set it per call. CoT billed as output |
| GPT-6.1 Sol (max effort) | OpenAI | Reasoning | $2.00 | $10.00 | $0.10 | 1M | Reasoning tokens billed as output; ~52 tok/s at max. AA index 52 at max vs 48 at medium — pay for max only when it moves your task |
| GPT-6.1 Sol | OpenAI | Mid-Tier | $2.00 | $10.00 | $0.10 | 1M | DevDay (Sep 29). Near-Astra agentic coding at 1/5 the price; 95%-off cache reads |
| Claude Sonnet 5.5 | Anthropic | Mid-Tier | $2.00 | $10.00 | $0.20 | 1M | Sep 28. AA 56; run at medium/high — max effort costs $7.60 per AA task |
| Claude Sonnet 5 | Anthropic | Mid-Tier | $2.00 | $10.00 | $0.20 | 1M | Intro price became permanent — the planned Sep 1 rise to $3/$15 never happened |
| GPT-5.6 Sol | OpenAI | Mid-Tier | $4.00 | $20.00 | $0.40 | 1M | Previous flagship. Cut from $5/$30 on Aug 21 — promo guaranteed only to Nov 21, 2026 |
| Mistral Medium 3.5 | Mistral | Mid-Tier | $1.50 | $7.50 | — | 256K | Open weights, EU-sovereign default; runs disconnected on Azure Local |
| GLM-5.3 | Z.ai | Mid-Tier | $1.40 | $4.40 | $0.26 | 1M | Open weights (MIT-style), AA 45. Flat across 1M; third-party hosts from ~$0.91/$2.86 |
| DeepSeek V4 Pro (0813) | DeepSeek | Mid-Tier | $1.32 | $3.96 | $0.044 | 1M | Peak-hour list price; off-peak $0.66/$1.98. Raised Aug 16 — now beaten by the cheaper V4.1 Flash |
| Muse Spark 1.2 | Mid-Tier | $1.25 | $4.25 | — | 1M | Meta's new proprietary coding model (beta, via Muse Code, Aug 5) | |
| Gemini 3.8 Flash | Mid-Tier | $0.75 | $3.75 | $0.075 | 1M | Intro price to Dec 31, 2026 → $1.50/$7.50 in 2027. AA 41; very verbose | |
| MiMo-V2.6-Pro | Xiaomi | Mid-Tier | $0.435 | $0.87 | $0.0036 | 1M | Open weights, highest open-weight AA score (46) — ~$0.13 per AA task |
| Qwen3.7-Plus | Alibaba | Mid-Tier | $0.40 | $1.60 | $0.04 | 1M | Alibaba's current Plus tier; >256K prompts $1.20/$4.80 |
| Claude Haiku 4.5 | Anthropic | Fast & Cheap | $1.00 | $5.00 | $0.10 | 200K | Fastest Claude; 4,096-token cache minimum. Haiku 5.5 announced |
| Grok Build 0.1 | SpaceXAI | Fast & Cheap | $1.00 | $2.00 | $0.20 | 256K | Beta; cheapest current Grok (Grok 4.1 Fast was retired May 15) |
| Gemini 3.5 Flash-Lite | Fast & Cheap | $0.30 | $2.50 | $0.03 | 1M | Cheapest current-generation Gemini; ~350+ tok/s | |
| DeepSeek V4.1 Flash | DeepSeek | Fast & Cheap | $0.30 | $1.20 | $0.006 | 1M | MIT weights (Sep 10). Peak list price — off-peak is half ($0.15/$0.60). The old $0.14/$0.28 card is gone |
| GLM-5.3-Flash | Z.ai | Fast & Cheap | $0.15 | $0.50 | $0.03 | 1M | MIT open weights, multimodal; AA 42 at ~1/9 of GLM-5.3's price |
| Qwen3.8-Flash | Alibaba | Fast & Cheap | $0.15 | $0.47 | $0.016 | 1M | Cheap multimodal workhorse; no peak pricing |
| GPT-6 Luna | OpenAI | Fast & Cheap | $0.10 | $0.50 | $0.01 | 1M | The new nano-tier default (Sep 22): ~50% below GPT-5.6 Luna, a cut OpenAI calls permanent; replaces GPT-5.4 Nano (deprecated Oct 1) |
| Qwen3.8-27B | Alibaba | Open Source | Free* | Free* | — | 262K | Apache 2.0; one 24GB GPU at 4-bit. *Self-host or free OpenRouter tier (rate-limited) |
| Gemma 4 (31B Dense) | Open Source | Free* | Free* | — | 256K | Apache 2.0; weights refreshed Jul 15; 1B+ Gemma downloads | |
| Muse Glimmer | Open Source | Free* | Free* | — | — | Apache 2.0, US-origin, ~30B agent-tuned; runs on one 24GB GPU (Aug 10) | |
| MiniMax M3 | MiniMax | Open Source | $0.30 | $1.20 | $0.06 | 1M | Open weights; prompts >512K bill 2×; 512K max output |
| Qwen3.8 2.4T-A95B | Alibaba | Open Source | Free* | Free* | — | 262K | Open Max-class twin (custom licence, $50M revenue gate); multi-node clusters. *Self-host |
Takeaway: GPT-6 Luna now undercuts DeepSeek's peak-hour price on every workload, and on agent loops the cache-read rate matters more than the headline price — without caching, the same Opus 5.5 workload costs ~$72,000/mo.
These are proven, production-tested techniques. Combined, organisations routinely achieve 40–80% cost reduction. Start with the high-impact ones — they require minimal engineering effort.
Providers cache the key-value matrices of repeated prompt prefixes. This is the single highest-impact optimisation — up to 90–97.5% cheaper on cached input tokens with minimal code changes.
cache_control parameter for explicit breakpoints. Target 80–95% hit rate. October 2026 note: cache reads now cost anywhere from 1/40 of list input (Claude Fable 5.1, $10 → $0.25) to 1/10 (GPT-6 Astra, $10 → $1.00) — a 4× spread at the same $10 sticker, and 10× in absolute terms from GPT-6.1 Sol's $0.10 to Astra's $1.00. On long agent loops the cache rate, not the list price, sets the bill — OpenAI says its GPT-6 caching upgrade more than halved freshly processed tokens across GitHub Copilot.
The 2026 default architecture. Send 70% of traffic to nano-tier (GPT-6 Luna / DeepSeek V4.1 Flash / Qwen3.8-Flash / Gemini 3.5 Flash-Lite) for classification + simple tool calls, 25% to mid-tier (Claude Sonnet 5.5 / GPT-6.1 Sol / Gemini 3.8 Flash / GLM-5.3) for structured reasoning, and only 5% to frontier (Claude Opus 5.5 / Fable 5.1 / GPT-6 Astra) for ambiguous planning. Cost runs at ~23% of sending everything to Claude Opus 5.5 ($5,439 vs $23,550/mo in the agentic scenario above), with statistically indistinguishable quality on 200-pair golden sets. The October twist: route on two axes — tier AND effort. One independent test found GPT-6 Sol at maximum effort only narrowly beats GPT-6 Astra at minimum effort while costing more per task, and Claude Sonnet 5.5 at medium costs ~1/13 of its max-effort run. Sakana's Fugu Ultra v2.0 ($5/$30) even sells routing as a model.
gen_ai spans BEFORE turning on routing. Pin exact gateway versions and hashes — poisoned LiteLLM 1.82.7/1.82.8 were live on PyPI for ~40 minutes on Aug 13 and stole cloud, SSH and Kubernetes credentials.
Every message in a multi-turn chat is resent on every call. A 40-turn conversation carries 25,000+ input tokens — the largest invisible cost driver. Sliding window truncation + summarisation eliminates this.
Instead of caching exact strings, semantic caching uses vector embeddings to serve cached responses when queries are semantically similar (≥0.85–0.95 cosine similarity). Avoids the API call entirely for repetitive queries.
Verbose prompts don't produce better outputs — they just cost more. Filler phrases like "You are a helpful, professional, knowledgeable assistant" add tokens with zero information gain. Telling a model to "be concise" reduces output tokens by 57–59%.
Output tokens typically cost 3–5× more than input. Setting max_tokens limits and requesting structured JSON instead of prose cuts the most expensive part of every call.
max_tokens parameter per call typemax_tokens as a hard upper bound for each use-case tier.
RAG pipelines often dump entire document chunks into context — including sentences irrelevant to the query. Pre-filtering retrieved chunks to only query-relevant sentences cuts RAG input tokens by 50–80% with maintained accuracy.
Most providers offer 50% discounts for asynchronous batch API calls. Non-real-time workloads — nightly reports, document analysis, bulk classification — are ideal candidates with zero quality trade-off.
Reasoning models — Claude Opus 5.5 (thinking can't be disabled; effort default medium), Claude Fable 5.1 (always on), Claude Sonnet 5.5 (defaults to high; thinking off only via between_tools), GPT-6.x (reasoning effort) — generate internal chain-of-thought tokens before responding: billed but invisible. CoT can be 3–10× the visible output. For a FAQ bot or classifier, you may be paying 5–20× more than necessary.
Fine-tuning embeds task knowledge directly into the model, eliminating the need for lengthy few-shot examples in every prompt. For stable, high-volume tasks, fine-tuned smaller models can outperform generic larger ones at a fraction of the cost.
* Savings are on applicable token spend per technique — not additive across all calls. Most production systems achieve 70–85% total spend reduction by combining prompt caching + 70/25/5 routing + context management + agentic budget caps. Source: Obvious Works 2026, Redis LLMOps Guide, FinOps Foundation State of FinOps 2026.
In the chatbot era, one user message = one API call. In the agentic era, one user message can fan out into 5–50 LLM + MCP tool calls — and a Reflexion loop running 10 cycles consumes 50× the tokens of a linear pass. This is the dominant tokenomics story of 2026, and it's why 73% of enterprises blew past their AI budget.
1 call · 500–2K tokens · $0.001–$0.02 per task
3–8 calls · 5K–15K tokens · $0.05–$0.50 per task
15–30 calls · 50K–200K tokens · $1–$8 per task
50+ calls · 200K–1M+ tokens · $5–$50 per task
Anyone who modelled costs on chatbot benchmarks is now 5–30× over budget. Microsoft Research: an unconstrained coding agent costs $5–$8 to resolve a single SWE-bench issue.
A self-critique loop running 10 cycles consumes 50× the tokens of a single linear pass. Without circuit breakers, a stuck ReAct loop can burn $50/min on frontier models.
Episodic memory writes (MemOS, Mem0, Zep) and skill-doc generation (Hermes Agent pattern) consume output tokens. Budget for them in OTEL gen_ai.tokens.write spans.
The 2026 SLO unit: cost-per-resolved-task, not cost-per-token. September's proof points: GPT-6 Astra costs 2.5× GPT-5.6 Sol per token but finishes OSWorld tasks in ~47% less time with fewer tokens (vendor); GPT-6.1 Sol costs $5.47 per Terminal-Bench Science task vs Astra's $23.80; and Grok 4.7 doubled cost per task vs Grok 4.6 at the identical $2/$6 price. Agent SREs alarm when OTEL cost.per_task p95 exceeds 2× baseline, auto-rolling back via Argo Rollouts.
FinOps Foundation State of FinOps 2026: Average enterprise AI budget grew from $1.2M (2024) → $7M (2026). 73% of enterprises report AI costs exceeded original budget projections. Fortune 500 firms report monthly inference bills in the tens of millions. The root cause in nearly every case: chatbot-era cost models applied to agentic-era workloads. Gartner (Sep 2026) raised its 2026 worldwide AI spending forecast to $2.7T (+49.5% YoY), and Salesforce alone has committed ~$300M to Anthropic tokens in 2026.
Beyond prompt-level changes, these infrastructure techniques address token costs at the compute layer — particularly relevant for self-hosted deployments and high-scale agentic systems.
For self-hosted deployments, the KV (key-value) cache is the dominant memory bottleneck. Recent research shows 70–90% memory reduction is achievable with minimal accuracy loss — enabling longer contexts on the same hardware.
Pair the target model with a lightweight draft model that proposes multiple tokens simultaneously. The target model verifies the batch in a single forward pass — achieving 2–5× faster generation with identical output quality.
Multi-agent systems can consume 4–15× more tokens than single-agent calls if not carefully orchestrated. Parallel execution, tool fusion, and model tiering within agent graphs dramatically reduce token overhead.
Quantisation reduces model weight precision from FP16 to INT8 or INT4, cutting memory requirements by 2–4× with minimal quality loss. Enables running larger models on cheaper hardware.
Where your model runs shapes per-token economics as much as which model you choose. At low volumes, API access wins. At scale, the economics flip decisively. Two-thirds of enterprises are now repatriating AI workloads on-premise (Finout 2026).
NCP becomes cheaper per-token than API access
AI factory per-token cost drops below API access
AI factory beats both API and NCP on per-token TCO
AI factory delivers 50%+ savings vs API at equivalent token volumes
Source: Deloitte "The Pivot to Tokenomics" — AI Economics Report 2025
Largest direct cost. NVIDIA HGX B200 GPUs, high-bandwidth memory, accelerators. Dominant cost at 10B token scale.
AI GPU racks draw 250–300kW vs 10–15kW for standard servers. Liquid cooling required at scale.
AI frameworks, orchestration tools, MLOps platforms, compliance tooling, enterprise support.
InfiniBand/NVLink GPU interconnects. High-bandwidth switches. Contributes 10–20% of TCO typically.
You cannot optimise what you cannot see. AI FinOps is the emerging practice of applying cloud financial governance discipline to token-based spending — and it's now the #1 priority for FinOps teams in 2026.
State of FinOps 2026 (FinOps Foundation): AI is now the fastest-growing new category of enterprise spend — average AI budget jumped from $1.2M (2024) to $7M (2026), and 73% of enterprises report AI costs exceeded original budget projections. 98% of respondents now use FinOps to manage AI spend (up from 31% in 2024). 33% named FinOps for AI as their #1 current or future priority — ahead of all others. New 2026 metric: cost per thought and token budget per project have replaced raw token counts as the unit economics of choice.
gen_ai spans to CFO-grade unit economics. Enterprise FinOps standard.Everything above, distilled into an operating doctrine — October 2026 edition. The through-line: architecture beats prompt tweaks, measurement beats intuition, and cost-per-completed-task beats every per-token metric. A cheaper path that silently drops quality is the most expensive mistake in this entire guide.
gen_ai spans; tag every call with model, team & use-case. You cannot cut what you cannot seegen_ai spans, tag by model/team/use-case, stand up a cost-per-task dashboard. No optimisation yet — you need a before picturecost.per_task p95 > 2× baseline; re-shop the pricing table monthly; re-run the golden set on every model or prompt changeEstimate your monthly API spend and see how the optimisation levers reduce it in real time. Toggle each lever to build your optimisation roadmap.