Alternative methods for evaluating the equivalence of measurement systems.

Abstract

Aim: Reports are common in the orthodontic and dental radiographic literature comparing measurement systems. Typically, such comparisons are made using differences in mean scores across methods, correlation coefficients, or Bland-Altman plots. These methods are subject to known limitations, including an inability to detect bias.

Methods: A hypothetical dataset was created to contain a small, common random variance and two types of bias to compare three alternative measurement systems against a common standard. One comparison included only random error and no bias. Two types of bias were investigated: systemic overestimation on the part of one measurement system and the more complex case of overestimation in one part of the range and underestimation in the other.

Results: Each of the commonly used methods for comparing alternative measurement systems was inadequate for detecting one or the other type of bias. The traditional regression analysis, by contrast, provided a useful characterization of the alternative measurement systems, including quantification of the nature of bias.

Conclusion: Regression analysis can reveal biases masked in the common comparison of means, correlation coefficients, and Bland-Altman plots for evaluating alternative measurement systems and thereby improve confidence in clinical applicability of research.