# Wake-word and stop-phrase detection for VAHA: dataset, validation accuracy and first field check

- Version: v1.0, published 21 October 2026
- Written by: Ritesh Bonthalakoti
- Reviewed by: Bharat Meda
- Domains: On-device AI
- Readiness: TRL 3, experimental proof of concept
- Tested on: An Edge Impulse keyword-spotting model (MFCC features, Keras classifier, three classes), on the platform's validation split and in live classification; the shipped model on an Arduino UNO Q with a USB lavalier microphone, in one bathroom check that was not logged
- Standards: None for keyword-spotting accuracy; ISO/IEC/IEEE 29119-3 loosely for structure; no accredited testing
- Full paper: [PDF, 17 pages, 0.8 MB](https://dharanova.com/whitepapers/vaha-wake-word-stop-phrase-detection.pdf)

## Summary

VAHA is Dharanova's offline voice-capture device. It starts recording when it hears the wake word "Marvin" and stops on a stop phrase (class im_done) or on silence. Detection uses an Edge Impulse keyword-spotting model, MFCC features and a Keras classifier, that runs on an Arduino UNO Q. This paper reports what the model was trained on and how well it classifies. The training project holds 8,466 one-second samples in three classes, split 80/20 into training and test. The marvin class comes from the public Google Speech Commands dataset and bg was imported from another Edge Impulse project; im_done was recorded for VAHA. On the platform's validation split the int8 model scored 97.6 % (95 % interval 96.6–98.3 %, calculated), below the 98.7 % quoted earlier for this model. Three unseen test samples, one per class, were all classified correctly with over 99.9 % confidence. A 34-trial check in a bathroom passed every trial, but individual trials were not logged. Every measured accuracy so far comes from the cloud platform, not from the shipped model on the device. The work is an experimental proof of concept (TRL 3).

## Key findings

- **Accuracy.** 97.6 % on the validation split, int8 model (96.6–98.3 %, calculated). Recall: bg 94.7 %, im_done 99.3 %, marvin 97.4 %.
- **Earlier figure.** The 98.7 % quoted earlier for this model lies outside that interval. The run it came from has not been identified.
- **Dataset.** 8,466 samples. marvin is public Google Speech Commands data and bg was imported from another project; im_done was recorded for VAHA.
- **Unseen samples.** 3 of 3 correct at 99.9 % or more, 19.9 to 25.0 points above the firmware thresholds.
- **Field check.** 34 of 34 passed in a bathroom, without a per-trial log. Logged, the same result would show at least 91.6 %.
- **Readiness.** TRL 3. Next: the shipped model on the UNO Q over the 1,689 test samples.

## Figures shown on the page

- Figure 2: The VAHA audio hardware with the enclosure open. Photo of the open enclosure on a desk, numbered: 1, the speaker, with a PAM8403 amplifier and driver, for chime and speech output and not in the detection path; 2, the Arduino UNO Q, which runs new-marvin.eim and the wake and stop logic; 3, the 3.5 mm microphone input jack, part of the audio input path; 4, the USB lavalier microphone, the input the model listens on, 48 kHz mono.
- Figure 1: The detection path. Audio is classified in 1,000 ms windows, a new one every 255 ms (3.92 per second, calculated). A single window above threshold is enough to start or stop recording. Flow diagram of the detection path on the device. The USB lavalier microphone, 48 kHz mono, feeds a 1,000 ms window, a new one every 255 ms; MFCC spectral features of the window; a Keras classifier with three classes; and a confidence per window for marvin, im_done and bg. The model input is 16 kHz, and the resampling step is not yet characterised. Wake: marvin at 0.75 or more in one window (WAKE_CONSEC = 1) starts recording. Stop: im_done at 0.80 or more in one window (6-frame window), or silence, stops recording. On the device this runs as new-marvin.eim on the Arduino UNO Q, called by the VAHA backend.
- Figure 4: The dataset by class and split, with what is known of each class's source. Bar chart of 8,466 samples by class, split into training and testing. marvin, 2,096 samples: public Google Speech Commands, identified from filenames. im_done, 3,148: recorded for this project and stored in it. bg, 3,222: imported from another project, original source not yet traced.
- Figure 5: Validation confusion matrix for the int8 model, row-normalised. The diagonal is each class's recall. Three-by-three matrix of true class against predicted class, each row summing to 100 %. bg: 94.7 % bg, 3.3 % im_done, 2.0 % marvin. im_done: 0.7 % bg, 99.3 % im_done, 0.0 % marvin. marvin: 1.9 % bg, 0.6 % im_done, 97.4 % marvin. F1 scores: bg 0.96, im_done 0.99, marvin 0.97.
- Figure 6: What each piece of evidence can show about accuracy at 95 % confidence (calculated). The field-check bound treats the 34 passes as logged results. Interval chart of classification accuracy from 30 to 100 %, with the 98.7 % reported in the product README marked. Platform validation, int8 model, about 1,355 samples: 96.6 to 98.3 %. Live classification, 3 of 3 correct in the cloud: at least 36.8 %. Bathroom field check, 34 of 34 passed, summary only: at least 91.6 %.
- Figure 8: Path from this work to TRL 4. Five steps in sequence. 1, set limits first: accuracy, missed wakes and false wakes per hour, fixed before testing. 2, pin the model: record which run, and int8 or float32, ships as new-marvin.eim. 3, test what ships: run new-marvin.eim on the UNO Q over all 1,689 test samples. 4, log field trials: a per-trial log, at least 59 trials per phrase, plus hours of bathroom noise. 5, provenance: source and licence for every class, bg included. These lead to TRL 4, the shipped detector validated in the lab against defined criteria. Then: define the relevant environment (bathroom noise, echo, distance) and test there (TRL 5).

## Related reading

- [Machine learning on a microcontroller](https://dharanova.com/learn/tinyml.md): background on Learn
- [Quantisation: making models fit](https://dharanova.com/learn/quantisation.md): background on Learn
- [On the device or in the cloud?](https://dharanova.com/learn/on-device-vs-cloud.md): background on Learn
- [The one room a phone does not follow you into](https://dharanova.com/arduino-physical-ai-challenge-2026.md): an engineering note from our team
- [Speech recognition that runs on the device](https://dharanova.com/on-device-speech-recognition.md): an engineering note from our team
- [On-device speech-to-text for VAHA: the model, where transcription happens, and a first accuracy test](https://dharanova.com/whitepapers/vaha-on-device-speech-to-text.md): another whitepaper

This paper documents component-level work in the lab. It is not a product specification.