A Serious Game for Soft Skills Assessment in Human Resources: Cross-Sectional Within-Participant Convergent Validity Study.

Abstract

Background: Soft skills are increasingly assessed in human resources, but commonly used methods (eg, interviews and self-report questionnaires) have well-known limitations. Serious games have been proposed as a complementary assessment format because they can standardize administration, embed assessment in interactive scenarios, and capture behavioral traces. However, evidence for their psychometric validity remains limited and heterogeneous. Establishing convergent validity against well-established reference instruments is a key step in supporting their use as assessment tools.

Objective: This study aimed to evaluate the convergent validity of Yuzu, a serious game that assesses (1) active listening via a gamified questionnaire inspired by the Active-Empathic Listening Scale (AELS), (2) decision-making under uncertainty via a gamified adaptation of the Iowa Gambling Task (IGT), and (3) teamwork style via dialogue choices inspired by the SYMLOG (System for the Multiple Level Observation of Groups) model.

Methods: We conducted a cross-sectional, within-participant convergent validity study in France with 39 adults (n=23 women; mean age 27.79, SD 7.96 y). Participants completed a single laboratory session on a desktop PC with headphones (Yuzu build v2, developed by Yuzu). Participants completed the 3 Yuzu modules and the corresponding reference instruments, administered separately (the AELS, an online IGT via PsyToolkit, and a simplified SYMLOG questionnaire). Primary outcomes were the associations between Yuzu and reference scores for each construct (active listening total score, IGT exploitation-phase net score, and SYMLOG dimension scores). Convergent validity was examined using Spearman correlations (2-sided α=.05). Agreement was additionally examined using Bland-Altman analyses for active listening and equivalence testing using two one-sided tests (TOST) for IGT net scores.

Results: At α=.05, active listening showed strong convergence between Yuzu and the AELS total score (Spearman ρ=0.890, 95% CI 0.804-0.948; P<.001), with minimal systematic bias (mean difference of 0.024, 95% CI -0.031 to 0.078). Decision-making scores were statistically equivalent across modalities based on TOST. The mean net score difference was 1.35 (90% CI -1.22 to 3.93), within the equivalence bounds [-5,+5] (TOST lower: P=.001; TOST upper: P=.01). Teamwork dialogue scores did not converge with the SYMLOG dimensions (dominance: ρ=-0.070, 95% CI -0.420 to 0.280; P=.69; positivity: ρ=0.082, 95% CI -0.266 to 0.418; P=.64; task orientation: ρ=0.134, 95% CI -0.260 to 0.494; P=.44), consistent with a ceiling effect toward cooperative choices.

Conclusions: This study provides convergent validity evidence for 2 complementary assessment modalities embedded in a single serious game, showing that a gamified AELS-inspired module and a gamified IGT adaptation can closely match established reference measures while supporting standardized administration. In contrast, the dialogue-choice teamwork module showed limited sensitivity and no convergence, suggesting that interpersonal profiling in serious games may require more discriminating scenario design and stronger controls for social desirability. Unlike many previous studies that evaluated a single game component, this study provides a module-by-module convergent validity blueprint within a single platform by using matched reference instruments, thereby informing both research and human resources deployment.