Replicability and Validity of a New Artificial-Intelligence Assessment of Posttraumatic Stress Disorder From Patient Language: A Sequential Evaluation With Model Preregistration.

Abstract

Artificial intelligence (AI) shows promise in identifying psychopathology through language, but replicability in AI models remains challenging. We develop an AI-based language assessment of posttraumatic-stress-disorder (PTSD) severity and introduce the sequential evaluation with model preregistration to rigorously evaluate its validity and replicability. This design includes two phases: development with preregistration and evaluation. Data included development (N = 1,437) and prospective (N = 346) samples, in which participants described their lives during automated interviews. In the prospective sample, preregistered models correlated with PTSD CheckList scores (r = .38, p < .001) and converged with PTSD diagnosis (area under the curve [AUC] = .76; outperforming demographics and trauma exposures: AUC = .61, p < .01). We found that for each standard-deviation increase, mental-health-care expenditure rose by $696.50 (p < .001). Our preregistered PTSD model assessments are replicable in prospectively collected clinical data and showed external validity against expense criteria. With further development, such models can be used to screen for PTSD or monitor treatment response, especially in telehealth or automated interviews, in which deployment can be seamless.