Benchmark
A self-hosted model matched cloud quality at a tenth of the latency.
Instead of arguing self-hosted versus cloud, I ran the numbers on a real workload. That meant 1,200 vision-analysis calls across six backend arms, on the same scenes, judged by two independent LLM judges. The data and the script are public, so you can re-run the whole thing yourself.
The short version
The finding, first
I hosted a 30-billion-parameter vision model on my own hardware and pointed the same job at it and at the cloud. It matched the best cloud model's answer quality, close enough that the gap sits inside statistical noise.
The difference was speed. The self-hosted model returned in about 0.77 seconds. The cloud models took 3.6 to 7.8 seconds for the same work, which is five to ten times slower. And the self-hosted one cost nothing per call, on hardware it was already running.
That is one workload, not a law about yours. It is a measured result with the raw data attached, so you can check it. Everything below is how it ran and what each number means, so you can judge whether it transfers to your case, or clone it and find out.
Results
Four numbers that carry it
From 1,200 calls across six backend arms: three cloud models, a no-extended-thinking control, and two self-hosted, on the same scenes.
Method
How it was run
The benchmark ran on a real product I had shipped. I had built a device that watches a room and works out what matters in it. Bare-metal firmware on an STM32U5 that sleeps and wakes itself, an agentic backend, and a vision model doing the actual analysis. So the workload I measured against was a genuine one.
Six backend arms went in. Three cloud models, one of them run a second time with extended thinking switched off as a control, and two models I hosted myself. Each arm saw the same scenes.
I ran it many times over, so one lucky or unlucky call could not swing the result. That came to 1,200 vision-analysis calls in total. For every call I recorded latency and cost.
Answer quality was scored separately, on a 0 to 9 scale, by two independent LLM judges on a shared subset of the runs. The full write-up is results/findings_v3.md in the repo. The runner and the raw rows sit right beside it, so the whole thing re-runs.
Findings
What the numbers said
Speed was not close. The self-hosted 30B model answered in about 0.77 seconds. The cloud models took 3.6 to 7.8 seconds for the same work, which is five to ten times slower, and they billed per call on top of it.
Quality was the surprise. On the 0-to-9 scale the self-hosted model scored 6.05. The best cloud model scored 6.60. That is barely half a point between them, judged blind by two independent LLM judges on the same scenes, so the gap sits inside the noise rather than in the answers.
The expensive option's headline feature bought nothing here. The cloud model's extended-thinking mode added no measurable quality on this workload. It just charged more tokens and more time for the privilege.
The cheap seat has a floor, though. The smaller model I hosted, a 3B, was too small for the job. It scored weakest on quality and could not use tools at all. Worth knowing before you reach for it to save money.
Why this matters
Measure before you commit a model
The expensive default bought nothing I could measure on this job. The cheapest option quietly failed at the thing that mattered. Neither surprise is something you can feel from the outside.
So measure before you put a model into production. The right choice is workload-specific.
A benchmark on your own traffic is cheap next to a year of the wrong inference bill. When I take on an AI-cost problem, this is the first thing I do: run your workload against the alternatives and let the numbers pick.