# On the device or in the cloud?

Where a model runs affects speed, privacy, cost, and what happens when the network is down.

Page: https://dharanova.com/learn/on-device-vs-cloud · Track: Infer · Level: Foundation · Facts last checked 7 October 2026

Diagram: Three rows. On the device: sensor to model to decision, with no network delay, raw audio and video staying local, and working offline. In the cloud: sensor to network to model to decision, so every decision needs a working connection. Safeguard, a confidence threshold: a bar from low to high confidence split by a threshold; results below it are ignored and results above it are acted on. A classifier always gives an answer, and scores are often too high, so the threshold should be set using real data.

## Key points

1. Running a model on the device avoids network delay, keeps raw audio and video local, and works offline. Running it in the cloud means every decision needs a working connection. [1][2]
2. A classifier always gives an answer, even for inputs unlike anything it was trained on. A common safeguard is to ignore results below a confidence threshold, but confidence scores are often too high, so set the threshold using real data. [3][4]

## Where to go next

- [Speech recognition that runs on the device](https://dharanova.com/on-device-speech-recognition.md): an engineering note from our team
- [The one room a phone does not follow you into](https://dharanova.com/arduino-physical-ai-challenge-2026.md): an engineering note from our team
- [On-device speech-to-text for VAHA: the model, where transcription happens, and a first accuracy test](https://dharanova.com/whitepapers/vaha-on-device-speech-to-text.md): a whitepaper from our lab
- [Wake-word and stop-phrase detection for VAHA: dataset, validation accuracy and first field check](https://dharanova.com/whitepapers/vaha-wake-word-stop-phrase-detection.md): a whitepaper from our lab

## Sources

1. [ML Kit: on-device machine learning](https://developers.google.com/ml-kit), Google
2. [LiteRT for Microcontrollers](https://developers.google.com/edge/litert/microcontrollers/overview), Google
3. [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks (Hendrycks & Gimpel, ICLR 2017)](https://arxiv.org/abs/1610.02136), arXiv / ICLR
4. [On Calibration of Modern Neural Networks (Guo et al., ICML 2017)](https://proceedings.mlr.press/v70/guo17a.html), PMLR