Assessing Threshold Ordering and Advancement in the Polytomous Rasch Model: A Probability-Based Framework.

Abstract

Rating scales are widely used to measure attitudes, personality traits, and other psychological constructs, yet their response categories may not function as intended. This paper introduces a probability-based framework for interpreting threshold ordering and threshold advancement in the polytomous Rasch model. Specifically, the framework clarifies why threshold ordering and threshold advancement by a useful amount matter for measurement and what is at stake when either requirement is violated. Within this framework, the paper proposes a probability-ratio criterion that links threshold distance to the relative probabilities of adjacent and nonadjacent response categories. The criterion provides researchers and practitioners with a simple, readily interpretable, and context-sensitive basis for determining the minimum threshold distance required to distinguish response categories with a desired probability contrast. The framework is illustrated theoretically and evaluated empirically using the rating scale model and the partial credit model, two parameterisations of the polytomous Rasch model, across a large set of instruments that employ common 4- and 5-point agreement and frequency scales. The empirical findings show that the same rating scale may function differently across instruments, even when administered to the same respondents, and that the central neutral response category in agreement scales warrants particular attention because it frequently exhibits problematic threshold functioning. Taken together, these methodological and empirical contributions advance the analysis of rating-scale functioning using the polytomous Rasch model.