Effects of a constructed response retest strategy on faking, test perceptions, and criterion-related validity of situational judgment tests

Abstract

This research proposes a faking-mitigation strategy for situational judgment tests (SJTs), referred to as the constructed response retest (CR-retest). The CR-retest strategy involves presenting SJT items in a constructed response format first, followed by equivalent closed-ended items with the same situation description. Two field experiments (N₁ = 733, N₂ = 273) were conducted to investigate the effects of this strategy and contrast it with a commonly used pretest warning message. Study 1 revealed that the CR-retest strategy was more effective than the warning message in reducing score inflation and improving criterion-related validity. Study 2 delved deeper by investigating the effects of the CR-retest strategy on applicant reactions in a 2 (with or without CR-retest strategy) × 2 (warning or control message) between-subjects design. The results showed that applicants reported positive fairness perceptions on SJT items with the CR-retest strategy. The CR-retest strategy was effective in reducing faking by evoking threat perceptions, whereas the warning message heightened threat and fear. Combining two strategies further decreased faking without undermining fairness perceptions. Overall, our results indicate that the CR-retest strategy could be a valuable method to mitigate faking in real-life selection settings.