Interobserver Agreement (IOA)

IOA tells you whether your measurement system is actually measuring what you think it's measuring. Two observers watch the same behaviour simultaneously and independently, then you compare their records. High agreement means the behaviour is defined clearly enough that different people see it the same way. Low agreement means your operational definition is too vague, the behaviour is hard to observe, or your data collection method is too complex for real conditions.

The threshold most commonly cited in the literature is 80%, but that number has limits. Eighty percent agreement on a behaviour that occurs very rarely can mask meaningful disagreement - two observers can agree on the absence of behaviour most of the time without ever agreeing on when it actually occurs.

IOA is also a proxy for procedural fidelity. If observers can't agree on what they're seeing, staff implementing the intervention probably can't either.