Applying Bayesian checks of cancellation axioms for interval scaling in limited samples.

Abstract

Interval scales are frequently assumed in educational and psychological research involving latent variables, but are rarely verified. This paper outlines methods for investigating the interval scale assumption when fitting the Rasch model to item response data. We study a Bayesian method for evaluating an item response dataset's adherence to the cancellation axioms of additive conjoint measurement under the Rasch model, and compare the extent to which the axiom of double cancellation holds in the data at sample sizes of 250 and 1000 with varying test lengths, difficulty spreads, and levels of adherence to the Rasch model in the data-generating process. Because the statistic produced by the procedure is not directly interpretable as an indicator of whether an interval scale can be established, we develop and evaluate procedures for bootstrapping a null distribution of violation rates against which to compare results. At a sample size of 250, the method under investigation is not well powered to detect the violations of interval scaling that we simulate, but the procedure works quite consistently at N = 1000. That is, at moderate but achievable sample sizes, empirical tests for interval scaling are indeed possible.