# Quantisation: making models fit

Storing weights as 8-bit integers instead of 32-bit floats makes a model about four times smaller. Quantising the activations too lets integer-only NPUs run it.

Page: https://dharanova.com/learn/quantisation · Track: Infer · Level: Deep dive · Facts last checked 7 October 2026

Read first: [Machine learning on a microcontroller](https://dharanova.com/learn/tinyml.md)

Diagram: Top: one weight as 32 squares for a 32-bit float and 8 squares for an 8-bit integer, about 4 times smaller. Middle: full integer quantisation also quantises activations so the whole model runs in fast integer arithmetic, usually with a small loss of accuracy; it feeds an integer-only NPU such as Arm's Ethos-U, and any operation the NPU cannot run falls back to the CPU. Bottom: post-training quantisation converts an already-trained model; quantisation-aware training simulates low precision during training and usually recovers more accuracy.

## Key points

1. Converting 32-bit floating-point weights to 8-bit integers cuts model size by about 4×. Quantising the activations too (full integer quantisation) lets the whole model run in fast integer arithmetic, usually with a small loss of accuracy. [1]
2. Post-training quantisation converts an already-trained model; quantisation-aware training simulates low precision during training and usually recovers more accuracy. [1]
3. Many embedded NPUs, including Arm's Ethos-U, only accelerate quantised integer models, so a float model has to be quantised first. Any operation the NPU cannot run falls back to the CPU. [2][1]

## Where to go next

- [On-device speech-to-text for VAHA: the model, where transcription happens, and a first accuracy test](https://dharanova.com/whitepapers/vaha-on-device-speech-to-text.md): a whitepaper from our lab
- [Wake-word and stop-phrase detection for VAHA: dataset, validation accuracy and first field check](https://dharanova.com/whitepapers/vaha-wake-word-stop-phrase-detection.md): a whitepaper from our lab

## Sources

1. [LiteRT: Post-training quantization](https://developers.google.com/edge/litert/models/post_training_quantization), Google
2. [Vela: compiler for Arm Ethos-U NPUs (README and supported operators)](https://gitlab.arm.com/artificial-intelligence/ethos-u/ethos-u-vela), Arm