Benchmark, part two
When self-hosting a model pays for itself
The benchmark covered quality and speed. This is the cost side, worked out from the same logged calls and July 2026 list prices.
Result
At low volume the API is cheaper
One call in my workload sends an image and gets a few hundred tokens back. At July 2026 list prices, a thousand of those calls cost between $1.24 and $4.92 on the cloud models I tested.
A GPU costs a flat amount every month, whether you use it or not. Renting a card that can run the 30B model starts at about $160 a month. Owning a used one costs about $60 a month, with the purchase spread over three years and electricity included.
Against the most expensive cloud model, owning a card pays off from about 12,000 calls a month. Renting pays off somewhere between 33,000 and 267,000 calls a month, depending on the card. Against the cheapest cloud model, renting never pays.
For Hesperus I self-hosted anyway, because the local model answered in 0.77 seconds against 3.6 to 7.8, and the photos never left the machine.
Numbers
The main figures
Cloud costs come from the benchmark's measured token medians at July 2026 list prices. Hardware costs are list prices from named providers on the same date.
The cloud side
A call costs at most half a cent
The benchmark logged the tokens of all 1,200 calls, so the cost per call is the logged tokens times each provider's list price.
The most expensive setup's median call sent about 340 tokens and got about 260 back, with the extended-thinking tokens billed as output. At the model's list rates ($3 per million in, $15 per million out) that is $4.92 per thousand calls, under half a cent each. The cheapest one came to $1.24 per thousand.
The image is billed as input tokens, and providers count the same image very differently. One setup's median call carried about 1,270 input tokens, almost all of them the image. At flash-class pricing of $0.50 per million input tokens, that call still costs about a tenth of a cent.
If you can wait up to a day for answers, batch APIs halve the price at all three big providers, and prompt caching cuts repeated context to a tenth of the input price.
The self-hosted side
The self-hosted side
My benchmark ran on a workstation card, an RTX 6000 Ada with 48 GB of VRAM. It served the 30B vision model and a small 3B side by side through llama.cpp. Renting the same thing in a datacenter costs about $1,314 a month (OVH's list price for the L40S, the card's 48 GB datacenter version).
The 30B model on its own needs less than that. Its quantized weights are about 19 GB, so it fits on a 24 GB consumer card, with less room for context and only one model at a time. That class of card rents for $160 to $290 a month on RunPod, Vast or a dedicated Hetzner server. I didn't benchmark on those cards, so treat that tier as an estimate.
A used 24 GB RTX 3090 traded at $750 to $1,100 to buy in the first half of 2026. Over three years that is about $25 a month. The card draws around 350 watts under load, which at Dutch household prices adds a few tens of dollars a month if it never idles. All in, about $60 a month.
Capacity isn't the limit. Even at 70 tokens a second, one card can serve hundreds of thousands of calls like mine a month.
The formula
The formula
Divide the monthly cost of the GPU by the cost of one cloud call. The result is the number of calls a month where the GPU starts paying for itself.
Against $4.92 per thousand calls, owning a card at $60 a month breaks even near 12,000 calls a month. Renting at $290 a month breaks even near 59,000, and the 48 GB datacenter card near 267,000. Against the cheapest cloud model, renting never breaks even.
Most break-even estimates you find online assume 70B-plus models on several datacenter GPUs, at $3,000 to $20,000 a month. A 30B model on one card costs an order of magnitude less, which is why the numbers here are so much lower.
Crossover
Below 12,000 calls a month, pay per call
Conclusion
When I'd self-host
At low or uneven volume, if a cheap cloud model is good enough, paying per token is cheaper.
Self-hosting makes sense with steady volume on hardware you own, when the latency difference matters to users, or when the data can't leave your servers. For Hesperus the last two decided it.
Sources
- Measured token medians, latencies, and the benchmark hardware (RTX 6000 Ada, llama.cpp)the benchmark's public repoaccessed 2026-07-04
- Anthropic list prices ($3/$15 per million tokens on the mid tier, $1/$5 budget tier)platform.claude.comaccessed 2026-07-04
- Google Gemini flash-class list prices ($0.50 per million input tokens) and image token countingai.google.devaccessed 2026-07-04
- OpenAI list prices and 50% batch discountdevelopers.openai.comaccessed 2026-07-04
- 24 GB GPU rental range, roughly $160 to $500 a month across tiersrunpod.ioaccessed 2026-07-04
- RTX 4090 marketplace rental floor and medianvast.aiaccessed 2026-07-04
- Dedicated 20 GB Ada GPU server at 234 euros a month plus a 114 euro setup feehetzner.comaccessed 2026-07-04
- L40S 48 GB at $1.80 an hour, about $1,314 a month continuousovhcloud.comaccessed 2026-07-04
- Used RTX 3090 street prices through H1 2026 ($750 to $1,100, rising)craftrigs.comaccessed 2026-07-04
- Measured inference power draw (roughly 280 to 410 watts on a 24 GB card under load)gigagpu.comaccessed 2026-07-04
- Dutch and German household electricity priceseuenergyprices.euaccessed 2026-07-04
- 30B-class throughput on 24 GB consumer cards (roughly 70 to 196 tokens a second measured)runaihome.comaccessed 2026-07-04