Is self-hosting an open LLM cheaper than the API?
It can be — but only above a break-even volume. Open-weight models like Llama and Mistral run on GPUs at a very low marginal cost per token, so at high, steady, always-on traffic self-hosting is the cheapest option by far. But a GPU bills by the hour whether it's busy or idle, and you take on reliability, scaling, and ops — so below the break-even, per-token API pricing wins. Here's the math to find your line.
Why it's a break-even, not a yes/no
The two pricing models are fundamentally different:
- API: pay per token. Cost scales perfectly with usage — zero traffic, zero cost. No idle waste, no ops.
- Self-host: pay per GPU-hour. Cost is fixed once the GPU is on, regardless of how many tokens flow through it. Cheap per token only when the GPU is well utilized.
So self-hosting's per-token cost depends entirely on utilization. A GPU serving a firehose of requests is cheap per token; the same GPU idling between sporadic requests is wildly expensive per token.
The break-even math
Self-hosting is cheaper when: monthly GPU cost ÷ tokens you actually push through it < API price per token
Work it in three steps:
- Monthly GPU cost. GPU hourly rate × 730 hours (a month of always-on). A single mid-range inference GPU rented in the cloud is often a few hundred to ~$1–2k/month depending on the card.
- Tokens it can serve. Throughput (tokens/second the model does on that GPU) × seconds/month × your realistic utilization. Utilization is the killer variable — 20% vs 80% changes the answer by 4×.
- Compare. Divide (1) by (2) to get your self-host cost per token, and compare to the API's per-token price for an equivalent-quality model.
Rough shape of the answer
| Your situation | Usually cheaper |
|---|---|
| Low or spiky traffic, prototype, unpredictable load | API |
| Moderate but bursty traffic | API (or serverless GPU) |
| High, steady, 24/7 volume at good utilization | Self-host |
| Data must stay on your own infra (compliance) | Self-host (for reasons beyond cost) |
The crossover is about sustained utilization, not raw request count. Two apps with the same monthly tokens land on opposite sides of the line if one runs flat 24/7 and the other spikes for two hours a day.
The hidden costs people forget
- Idle time. You pay for the GPU overnight and on weekends even if traffic drops to zero.
- Ops and reliability. Uptime, autoscaling, failover, monitoring, model updates — real engineering time the API includes for free.
- Peak provisioning. To handle spikes you either over-provision (idle cost) or degrade under load.
- Quality gap. If the open model you can host isn't as good as the API model, you may need a bigger GPU or accept lower quality — factor that in.
A middle path: serverless / on-demand GPUs
Serverless GPU platforms bill closer to per-second and scale to zero, which softens the idle-cost problem for bursty workloads. They cost more per active hour than a reserved GPU but remove the pay-for-idle penalty — often the best fit for medium, uneven traffic that isn't quite big enough for a dedicated box. Compare GPU hosting options →
Compare against API cost first
Before pricing GPUs, know your API baseline. The calculator gives you the per-token and monthly API cost to compare your self-host math against.
Open the calculator →Verdict
Self-host when you have large, predictable, always-on volume and can keep a GPU well utilized — or when compliance requires data to stay on your infra. Otherwise the API is cheaper and simpler, especially once you've applied the cheaper-tier and routing levers in how to cut your bill 50%+. For most teams, optimizing API usage beats taking on GPU ops.
FAQ
Is self-hosting an LLM cheaper than using an API?
Only above a break-even volume. GPUs bill by the hour regardless of use, so self-hosting wins at high, steady, well-utilized traffic; below that, the pay-per-token API is cheaper and simpler.
What is the break-even for self-hosting?
When monthly GPU cost ÷ tokens actually served drops below the API's per-token price. Utilization is the deciding factor — the same GPU is cheap per token at 80% load and expensive at 20%.