Different paths, same destination? Comparison of two approaches to developing situational judgment tests for cross-cultural competence

Abstract

Assessing cross-cultural competence (3C) remains a challenge, as traditional self-report measures often fail to capture the complexity of intercultural interactions and are vulnerable to validity concerns. Situational judgment tests (SJTs) offer a promising alternative by simulating real-world decision-making in intercultural contexts. This study develops and evaluates two distinct SJTs via two item development methods, work-sampling and construct-based, in measuring 3C. Using a within-person design, we examine their psychometric properties, including reliability, internal structure, face validity, susceptibility to social desirability bias, and criterion validity. Both SJTs demonstrate acceptable reliability and correlations with a self-report 3C measure, overseas life satisfaction, and sociocultural adaptation. However, the construct-based SJT appears more susceptible to social desirability and has lower face validity compared to the work-sampling SJT. Only the work-sampling SJT explained peer-rated multicultural team performance. We contribute to the refinement of 3C assessment by developing the SJTs that could serve as viable alternatives to self-report scales. Our findings also suggest that work sampling SJTs may offer certain advantages over construct-based SJTs in measuring 3C.