Accuracy of therapists' predictions of outcome in internet-delivered cognitive behavior therapy for depression and anxiety in routine psychiatric care.

Abstract

Objective: Early identification of failing psychological treatments could be of high clinical value, but therapists themselves have been found to be bad at predicting who will benefit or not. Previous research has some methodological limitations, and therapists' predictive accuracy has never been examined in internet-delivered treatments. Method: Therapists providing internet-delivered cognitive behavior therapy for depression, social anxiety disorder, and panic disorder in routine psychiatric care made outcome predictions for 897 patients during the fourth week of treatment. Therapists' accuracies were also compared to the accuracy of a simple statistical model and benchmarks for clinically acceptable/useful levels of accuracy from previous research. Results: Therapists were more accurate than chance, but their balanced accuracy was on average nine percentage points lower than the balanced accuracy of the statistical model (though confidence intervals often overlapped) and only in one case did the predictions reach the clinical acceptance and utility benchmarks. Therapists could predict on average 16% of the variance in outcome. Therapists were overly optimistic, predicting positive outcomes on average twice as often as they occurred. They differed in confidence in their predictions, though this did not affect how correct they were. Conclusions: Internet-delivered cognitive behavior therapy-therapists can often predict treatment outcomes better than chance, but generally not as well as the statistical model, and probably not accurately enough that they would be willing to act on their predictions, or that they could be used in an adaptive treatment strategy. Our previous findings suggest that patients would benefit from statistical monitoring and prediction tools in clinical settings. (PsycInfo Database Record (c) 2025 APA, all rights reserved).