A multilevel argument-based approach to validation and interpretation of safety climate scores

Abstract

We introduce Kane's Interpretation Use Argument approach to establishing a measure's validity to organizational scientists and extend it to multilevel constructs. First, we review five types of inferences (i.e., domain description, scoring, generalization, extrapolation and implication) that could be targeted by applied psychologists and management scholars, enumerate the types of analyses that fall under each of the inferences and describe how they help provide a clearer overview to support score use. We apply this framework to organize evidence related to a short and theory-driven scale that measures safety climate by developing six potential factor structures for safety climate scores, along with their meaning and interpretations, selecting items from the SOPS survey and analysing data from the 2021 and 2022 SOPS datasets, two large government surveys from the health care industry (N = 77,674 and 183,573, respectively) that feature a nested data structure on three levels. A shared construct model was the model that received the most empirical support. We conclude with a discussion of the implications of measuring safety climate using shared construct models, the limitations of the SOPS survey, and we trace a map for future efforts to constructing a validity argument.