Utilisation is the whole argument
You rent GPUs by the hour, not by the token. At 100% utilisation a rented GPU is extraordinarily cheap per token; at 10% it's four to ten times worse than an API. Real production traffic is spiky — nights and weekends are near-idle — so honest utilisation for a single-tenant deployment is usually 25–40% unless you're running batch workloads that can soak up the trough.
The costs that don't appear on the GPU invoice
Engineering time to deploy, monitor, patch, and upgrade. On-call for a service that now has no vendor SLA. Model evaluation every time you consider a new open-weight release. Redundancy so a single node failure isn't an outage. At small scale these dwarf the compute line, which is exactly why early-stage teams should stay on APIs.
- • Batch and offline jobs are the best self-host candidates — they fill idle capacity.
- • Spot/preemptible instances cut GPU cost 40–70% if your workload tolerates interruption.
- • Quantised models (int8/int4) can double throughput per GPU.
- • Keep an API fallback path for burst traffic instead of over-provisioning.
When self-hosting wins for reasons other than money
Data residency and compliance, guaranteed model version stability, no per-request rate limits, custom fine-tuned weights, and predictable fixed costs. These are legitimate reasons to self-host at volumes where the pure cost comparison is a wash — just be honest that you're buying control, not savings.
Related guides
Long-form playbooks on the same topic, written by the RevenueLab editorial team.
FAQ
At what volume does self-hosting an LLM become cheaper?
Typically somewhere between 1 and 10 billion tokens per month, depending on GPU pricing, utilisation, and which API tier you're comparing against. Below roughly 100 million tokens/month, APIs almost always win once you count engineering time.
How much does it cost to run an open-source LLM?
Two mid-range GPUs at $2.20/hour is about $3,200/month in rental alone, regardless of usage, plus engineering time. That fixed cost only makes sense if you're pushing enough tokens through it to beat the equivalent API bill.
What GPU utilisation should I assume?
25–40% for a single-tenant production service with normal daily traffic patterns. 60–80% is achievable if you can backfill idle hours with batch work. Assuming 90%+ is how self-hosting business cases get built and then quietly fail.
Does self-hosting include fine-tuning cost?
Not in this calculator — training is separate. Self-hosting does remove the hosted fine-tune inference premium, though, which can make a custom model considerably more attractive at volume.
What about spot instances?
Spot or preemptible GPUs cut rental cost 40–70%, but can be reclaimed with little notice. Great for batch inference and training, risky for user-facing latency-sensitive serving unless you have a fallback.
Should I self-host for data privacy?
It's a valid reason, but check the alternatives first — most major providers offer zero-retention and enterprise data-processing terms that satisfy common compliance requirements at a fraction of the operational burden.
How this calculator is built
Independently maintained
Written by Sam Doshi and the RevenueLab editorial team. We don't sell the data feeds this tool is built on.
Sourced from primary data
Benchmarks come from public AdSense / Stripe / IRS disclosures and reader-submitted data — never third-party "$X per view" claims. Full methodology.
Last editorial review
Reviewed on a rolling quarterly cycle. Dated reviews are published on the methodology record for each calculator.
Editorial standards
See our editorial policy and disclaimer. Results are estimates, not advice.