Large language models for depression assessment from brief daily diaries

Abstract

Abstract Closely tracking changes in depression is important for clinical research and practice. However, intensive longitudinal depression assessment relies on self-report surveys that have major limitations. Scoring depression from natural language with large language models (LLMs) could resolve these shortcomings. This study tested the validity of depression rated by LLMs from daily video diaries. We studied 108 participants selected for mental health treatment status (45% in treatment). Participants completed a clinical interview and 14-21 days of depression surveys and video diaries. Daily depression scores were estimated by LLMs from diary transcripts. LLM ratings converged with self-report depression within-person ( r  = .45;95% CrI[.40, .49]) and when averaged across days for an overall depression score ( r  = .61;95% CrI[.43,.75]). Averaged LLM ratings correlated with depression symptom severity ascertained by clinical interview ( r  = .53;95% CrI[.35, .68]). LLM-rated depression and self-report had similar profiles of associations with daily functioning variables (profile r  = .98 within-person; r  = .94 between-person). Multivariable regression results showed LLM ratings predicted daily functioning variables (partial $$\beta s$$ β s =|.12 | - | .28 | ) and interview-rated depression (partial $$\beta$$ β =.33;95% CrI[.07, .58]) over and above self-reports, suggesting LLMs capture clinically-relevant information that self-reports miss. Findings establish a strong foundation for a depression assessment approach that addresses shortcomings of self-reports while satisfying the unique demands of intensive longitudinal monitoring.