A Gamified Assessment Tool for Antisocial Personality Traits (Antisocial Personality Traits Evidence-Centered Design Gamified): Randomized Controlled Trial.

Abstract

Background: The traditional self-report instruments (eg, scales) used to measure antisocial personality traits are characterized by social desirability bias and fail to capture multidimensional behaviors (eg, manipulation and deception).

Objective: This study aimed to develop and validate an evidence-based design for a gamified assessment tool (Antisocial Personality Traits Evidence-Centered Design Gamified assessment tool; ASP-ECD-G) to measure 7 antisocial personality traits (manipulative, callous, deceptive, hostile, risk taking, impulsive, and irresponsible) as defined in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5).

Methods: This research featured a 3-phase evidence-centered design framework. Ontology development (study 1): semistructured interviews were conducted with 9 workplace professionals to translate the DSM-5 criteria into 24 observable workplace behaviors, which were integrated into a text-based game featuring 10 subscenarios, 34 interactive questions, and logic rooted in logical jumps to simulate real-world decision-making. Model construction (study 2): 6 machine learning models were trained by reference to a set of Personality Inventory for DSM-5 Short Form scores (n=286). The gated recurrent unit model, which uses 1-hot encoding to address nominal response data, was evaluated in terms of the root mean square error (RMSE), mean absolute error, criterion correlation (r), and test-retest reliability. Retest reliability was assessed using intraclass correlation coefficients based on 10 participants (1-month interval). Empirical validation (study 3): a 2×2 mixed design (n=148) was used to compare the gamified assessment with questionnaires under conditions involving incentives (ie, situations in which "rational results" led to increased payments).

Results: For model performance, the gated recurrent unit outperformed the alternatives, as indicated by the highest criterion correlation (r=0.850) and the lowest test RMSE (0.273); in particular, it excelled in moderate score ranges (1.5-3, RMSE≤0.377) and in resisting extreme value distortions (3.5-4, RMSE 0.854). Retest reliability was moderate to strong (intraclass correlation coefficients=0.776, P=.02). For validation findings, the gamified assessment was associated with higher levels of immersion (mean 7.628 vs 7.216; F147=14.259, P<.001) and interest (mean 7.095 vs 6.155; F147=47.940, P<.001), although it also elicited stronger negative emotions (mean 4.365 vs 2.473; F147=151.109, P<.001). Incentives reduced questionnaire scores (incentivized: 2.066 vs control: 2.201; F1=5.740, P=.02) but had no effect on gamified scores (P=.71), confirming resistance to manipulation.

Conclusions: By integrating evidence-centered design with gamified workplace simulations, ASP-ECD-G can provide more objective and ecologically valid measurements of antisocial personality traits, thereby supporting both research and organizational practice.

Trial registration: Open Science Framework (OSF) Registries tvg6x; https://osf.io/tvg6x.