Self-reports vs clinician ratings of efficacies of psychotherapies for depression: a meta-analysis of randomized trials.

Abstract

Aims: The comparability between self-reports and clinician-rated scales for measuring depression following treatment has been a long-standing debate, with studies finding mixed results. While the use of self-reports in psychotherapy trials is very common, it has been widely assumed that these tools pose a validity threat when masking of participants is not possible. We conducted a meta-analysis across randomized controlled trials (RCTs) of psychotherapy for depression to examine if treatment effect estimates obtained via self-reports differ from clinician-rated outcomes.

Methods: We identified studies from a living database of psychotherapies for depression (updated to 1 January 2023). We included RCTs measuring depression at post-treatment with both a self-report and a clinician-rated scale. As our main model, we ran a multilevel hierarchical meta-analysis, resulting in a pooled differential effect size (Δg) between self-reports and clinician ratings. Moderators of this difference were explored through multimodel inference analyses.

Results: A total of 91 trials (283 effect sizes) were included. In our main model, we found that self-reports produced smaller effect size estimates compared to clinician-rated instruments (Δg= 0.12; 95% CI: 0.03-0.21). This difference was very similar when only including trials with masked clinicians (Δg= 0.10; 95% CI: 0.00-0.20). However, it was more pronounced for unmasked clinical ratings (Δg= 0.20; 95% CI: -0.03 to 0.43) and when trials targeted specific population groups (e.g., perinatal depression) (Δg= 0.20; 95% CI: 0.08-0.32). Effect sizes between self-reports and clinicians were identical in trials targeting general adults (Δg= 0.00; 95% CI: -0.14 to 0.14).

Conclusions: Self-report instruments did not overestimate the effects of psychotherapy for depression and were generally more conservative than clinician assessments. Patients' perception of improvement should not be considered less valid by default, despite the inherent challenge of masking in psychotherapy.