Join the early list

Accuracy, precision and resolution

Accuracy, precision and resolution mean different things, and datasheets often blur them. Mixing them up gives readings that look precise but are wrong.

Foundation

Read firstWhat a sensor actually isAnalog-to-digital conversion
Accuracy, precision and resolution are three different things Four targets show readings that are accurate and precise, precise but not accurate, right on average but not precise, and neither. A scale with fine steps shows that a reading can be reported to one small step yet sit many steps from the true value. Calibration corrects offset and gain but not random noise. centre = true value dots = repeated readings Accurate and precise Precise but not accurate Right on average but not precise Neither scattered and off-centre Resolution the smallest change it can report High resolution does not imply high accuracy 1 step true value reading Offset, gain error, drift: far more than one step Calibration compares readings with a known reference Corrects steady errors: offset and gain Cannot remove random noise Hysteresis is hard to correct
Accuracy is how close readings are to the true value, precision is how closely they agree, and resolution is the smallest step the system can report.

Key points

  1. Accuracy is closeness to the true value; precision is how closely repeated readings agree with each other (under the same conditions this is called repeatability); resolution is the smallest change the system can report.1
  2. High resolution does not imply high accuracy: an ADC can report a reading to many digits while offset, gain error and drift make it wrong by far more than one step.2
  3. Calibration compares readings with a known reference and uses the result to correct steady errors such as offset and gain. It cannot remove random noise, and hysteresis (a reading that depends on whether the value is rising or falling) is hard to correct.12

Where to go next

Sources

  1. JCGM 200:2012 International vocabulary of metrology (VIM), 3rd edition · BIPM · Standard
  2. A Basic Guide to Bridge Measurements (SBAA532A) · Texas Instruments · Reference