Comparison between clinician and machine learning prediction in a randomized controlled trial for nonsuicidal self-injury.

Abstract

Background: Nonsuicidal self-injury is a common health problem in adolescents and associated with future suicidal behavior. Predicting who will benefit from treatment is an urgent and a critical first step towards personalized treatment approaches. Machine-learning algorithms have been proposed as techniques that might outperform clinicians' judgment. The aim of this study was to explore clinician predictions of which adolescents would abstain from nonsuicidal self-injury after treatment as well as how these predictions match machine-learning algorithm predictions.

Methods: Data from a recent trial evaluating an internet-delivered emotion regulation therapy for adolescents with nonsuicidal self-injury was used. Clinician predictions of which patients would abstain from nonsuicidal self-injury (measured using the youth version of Deliberate Self-harm Inventory) were compared to a random forest model trained on the same available data from baseline assessments.

Results: Both clinician (accuracy = 0.63) and model-based (accuracy = 0.67) predictions achieved significantly better accuracy than a model that classified all patients as reaching NSSI remission (accuracy = 0.49 [95% CI 0.41 to 0.58]), however there was no statistically significant difference between them. Adding clinician predictions to the random forest model did not improve accuracy. Emotion dysregulation was identified as the most important predictor of nonsuicidal self-injury absence.

Conclusions: Preliminary findings indicate comparable prediction accuracy between clinicians and a machine-learning algorithm in the psychological treatment of nonsuicidal self-injury in youth. As both prediction approaches achieved modest accuracy, the current results indicate the need for further research to enhance the predictive power of machine-learning algorithms. Machine learning model indicated that emotion dysregulation may be of importance in treatment planning, information that was not available from clinician predictions.

Trial registration: NCT03353961|| https://www.

Clinicaltrials: gov/ , registered 2017-11-21. Preregistration at Open Science Framework: https://osf.io/vym96/ .