LLM cost optimization: the complete 2026 guide
If your AI bill is metered — API tokens or GPU hours — most of it is probably avoidable. LLM cost optimization is the practice of paying less for the same output quality, and the gains are large: teams that have never optimized routinely cut their bill by half or more. This guide covers how pricing actually works, every lever that moves the number, how to choose a model, and when to self-host — with a free calculator and deep-dives for each topic.
Start with your own numbers
The calculator turns your usage into a per-model cost ranking and shows your routing savings in real time. Everything below explains what those numbers mean and how to move them.
Open the calculator →1. How LLM pricing actually works
You're billed per token, not per request. A token is roughly ¾ of a word. Every API call has two priced parts:
- Input tokens — everything you send: the system prompt, context, retrieved documents, chat history, and the user's message.
- Output tokens — everything the model generates back.
Each is priced per million tokens, and output typically costs 4–5× more than input. The formula is simple:
cost = (input_tokens ÷ 1,000,000 × input_price) + (output_tokens ÷ 1,000,000 × output_price), summed over every request.The practical consequence: your bill is driven by total tokens, which means volume × tokens-per-request, not the number of requests. A million tiny calls can cost less than a hundred thousand context-stuffed ones. For the full breakdown with examples, see how much the API actually costs and cost per 1,000 requests.
2. The levers that cut your bill
Seven levers, ranked by typical impact. Each is covered in depth in how to cut your bill 50%+; here's the map.
| Lever | Typical saving | Best for |
|---|---|---|
| Route easy work to a cheaper model | 40–70% | Almost everyone |
| Prompt caching | 20–50% input | Fixed system prompt / repeated context |
| Batch API (50% off) | 50% of eligible | Non-real-time work |
| Trim context sent | 10–40% input | RAG, long chat history |
Cap max_tokens | Variable | Preventing runaway output |
| Smaller model + better prompting | Large | High-volume repetitive tasks |
| Self-host open models | Large* | High, steady volume only |
They stack, and they overlap — you won't get every discount on every request. But the first lever alone reshapes most bills, which is why "cut it in half" is a conservative headline. Start with routing. *self-hosting saves below-zero under the break-even — see §4.
3. Choosing the right model and tier
The biggest cost decision isn't which vendor — it's which tier. Every vendor offers a budget, mid, and premium model, and within a vendor the budget option can be 10–25× cheaper than premium. Match the tier to the task:
| Task | Tier |
|---|---|
| Classification, extraction, tagging, short summaries | Budget |
| General chat, RAG answers, drafting | Mid |
| Hard reasoning, agents, code review, high-stakes output | Premium |
The discipline: test the cheaper tier on your own data and route down only where it passes your quality bar — don't assume, and don't reflexively reach for the biggest model. For a vendor-by-vendor and workload-by-workload breakdown, see Claude vs GPT vs Gemini.
4. API vs self-hosting
Open-weight models (Llama, Mistral, Qwen) run on GPUs you rent or own at a very low marginal cost per token. At high, steady volume this is the cheapest option. But a GPU bills by the hour whether busy or idle, and you take on reliability, scaling, and ops — so below a break-even volume the per-token API wins. Do the math before committing; we walk through the break-even in self-hosting vs API. Rule of thumb: self-hosting pays off only with large, predictable, always-on traffic.
5. Putting it together: an optimization checklist
A practical order of operations for a metered workload:
- Measure first. Put current usage into the calculator to see where the money goes.
- Route. Split traffic by difficulty; move the easy majority down a tier. Biggest, fastest win.
- Cache. If a large prompt prefix repeats, turn on prompt caching.
- Batch. Move everything non-real-time to the Batch API for 50% off.
- Trim. Tighten retrieval and cap chat history to shrink input tokens.
- Cap. Set a sane
max_tokensand prompt for concise output. - Re-measure. Confirm the savings and watch quality on your evals.
Automate the biggest lever
A routing layer applies step 2 for you — inspecting each request and sending it to the cheapest capable model, so you capture the saving without choosing a model per call.
Compare routing tools →6. Glossary
- Token
- The unit of text LLMs are billed on — roughly ¾ of an English word. "cost optimization" is about 3 tokens.
- Input / output tokens
- Input is everything you send (prompt, context, history); output is what the model returns. Priced separately, output ~4–5× higher.
- Context window
- The maximum tokens a model can consider at once (input + output). Bigger windows let you send more context — which costs more.
- Prompt caching
- Storing a repeated prompt prefix so subsequent requests read it at ~10% of the normal input price. Requires a byte-identical prefix.
- Batch API
- Running non-urgent requests asynchronously (results within ~1–24h) at 50% of standard price.
- Model routing
- Sending each request to the cheapest model capable of handling it, instead of everything to one premium model.
- Tier
- A vendor's price/capability band — budget, mid, or premium. The tier drives cost more than the vendor.
FAQ
What is LLM cost optimization?
Reducing what you pay for LLM API usage without hurting quality — chiefly by routing easy requests to cheaper models, caching repeated content, batching non-urgent work, and trimming context.
How are LLM API costs calculated?
Per token: (input ÷ 1M × input_price) + (output ÷ 1M × output_price), summed across requests. Output is priced ~4–5× higher than input.
What's the fastest way to lower an LLM bill?
Model routing — move the easy majority of traffic to a cheaper tier. Because tiers differ 10–25× in price, it typically cuts a bill by 40–70%.