Speech recognition that runs
on the device
· Ritesh Bonthalakoti
Every speech API works the same way. A microphone captures audio, the audio goes to a server, text comes back a moment later.
ElevenLabs, Sarvam, Google's — they are all good at it, and they all assume that trip is fine. For something sitting in a kitchen, it isn't.
Why not just call an API?
The round trip costs time. A device that answers a beat late reads as broken, and you cannot explain a user out of that impression.
The audio is somebody's home. Once it has left the building, whatever governs it is a promise rather than a fact.
And the network is not always there. A cloud-backed device does not slow down when the connection drops — it stops understanding anything at all. Those are the hours ours are built for.
What Ritesh tried
The constraint was the board, not the model. Plenty runs on a workstation with a graphics card. The question was what runs on something that could sit inside a product, which in this case meant an Arduino Uno Q.
Moonshine, Vosk, Whisper. Those were the local models worth taking seriously at that size, so he benchmarked them.
What worked
Faster-Whisper, the small model — about 244 million parameters — running on the board itself.
No API key. No round trip. Nothing to fail when the connection does.
What it doesn't prove yet
One board and one model is not a product, and we are not going to pretend it is. We are still experimenting with other combinations of board and model.
But we say that where a device of ours listens, the listening happens on the device and the audio stays there.