A meta-analytic examination of the criterion-related validity and cognitive loading associated with traditional and new situational judgment test paradigms.

Abstract

Traditionally, situational judgment tests (SJTs) include text-based situations and ask candidates to pick the best and worst response option. Over the years, various new SJT paradigms have been developed to increase validity, reduce incidental cognitive load, and/or lower costs. Examples are multimedia SJTs, constructed response SJTs, and single-response SJTs. First, this study investigates how the traditional SJT paradigm performs in terms of criterion-related validity by updating prior meta-analyses. Second, this study provides initial, meta-analytic evidence on the criterion-related validity and cognitive load of several new SJT paradigms. On the basis of 457 independent samples from 302 studies (total N = 244,257), results showed that SJTs were not only predictive of job performance but also of academic performance, organizational citizenship behavior, and counterproductive work behavior. SJT validity was higher in operational (i.e., applicant) settings than in nonoperational settings and could also be expanded to new content (e.g., emotion and cross-cultural management). In terms of comparing traditional and new SJT paradigms, multimedia SJTs had validity equal to text-based ones but lower cognitive load. Importantly, the multiple-choice modality did not significantly outperform written constructed response modalities in terms of validity, and the cognitive loading of webcam-constructed responses was even lower than that of other response modalities. The implications of these findings are among others discussed in the context of the cost effectiveness of SJTs vis-à-vis other selection procedures. (PsycInfo Database Record (c) 2026 APA, all rights reserved).