Sauti / self-hosted
Whisper large-v3 (self-hosted)
Open weights, run on our own hardware
WER
11.9%
95% CI 10.4%–13.6%
Rank
#4
of 12 measured
Latency
209.5s
mean, per clip
Cost
$0.00
per audio hour
Every other model on the board is a statistical tie with this one. Its confidence interval overlaps all 11 of them, so the rank above tells you where it landed, not that it is better or worse than anything else here.
What it is
OpenAI's open-weight Whisper large-v3 running on our own CPU bridge via faster-whisper. Same architecture as the hosted whisper-1, different operator.
Pick it when
When you can run your own inference. It placed in the upper half of the board and ahead of most hosted OpenAI models, at zero marginal cost per hour, and the weights are downloadable so nothing changes under you.
Think twice when
When latency matters at all. On our CPU bridge it is roughly two orders of magnitude slower per clip than the hosted APIs (see the latency figure above). That says as much about our hardware as about the model, but it is the honest cost of this option as we run it. The zero hourly cost is likewise marginal only: it is hardware we already own, and excludes capex and power.
Assessment written against the board of 2026-08-08. The figures above are read live from the current published snapshot (2026-08-08), so if those dates differ, trust the figures.
What the vendor publishes
OpenAI (Whisper paper) publishes no stated word error rate figure for this model on LibriSpeech. That absence is itself the finding. Source (read 2026-08-15).OpenAI publishes no LibriSpeech WER for large-v3 — it postdates the Whisper paper, whose Table 8 tops out at large-v2 (2.7 on test-clean). Do not attribute the large-v2 figure to v3.
How this was measured
250 clips (2.9 hours) of unscripted conversational English from People’s Speech (MLCommons, CC-BY), human-transcribed. Scored on a held-out split (N=77) never used to tune anything, through one fixed text normalizer, with bootstrap 95% confidence intervals. One corpus, one language, one modality: batch pre-recorded. It does not tell you how this model handles your accents, your domain vocabulary, or streaming.
Cost is self-hosted’s own published pre-recorded pay-as-you-go list price, read from their pricing page on 2026-08-08. Streaming and committed-volume rates differ.Marginal cost on hardware we already run. Amortised capex and power are excluded.