Assessing spoken discourse in aphasia using multimodal artificial intelligence.

Abstract

Discourse analysis can reliably predict real-world communicative success, yet is rarely implemented clinically, due to challenges in manually generating stimulus-specific main concept inventories (MCIs) and scoring patient narratives. We evaluated whether automatically derived macrolinguistic (main concept, sentiment) and microlinguistic (fluency, lexical, syntactic) features extracted from movie clip narrations can augment the clinical assessment of discourse in aphasia. The MCIs generated directly from each clip's video, audio and subtitles using a vision-language model approximated the main concepts healthy controls produced. Group classification (control: n=51; aphasia: n=54) based on the discourse features was excellent (AUC=0.95, 95% CI [0.90, 0.99]), with macrolinguistic features showing the largest group differences (main concept completeness: d=-1.84, 95% CI [-2.30, -1.38]) and strongest associations with aphasia severity (semantic distance to inventory: r=-0.79, 95% CI [-0.87, -0.66]). Findings support the feasibility and scalability of automated criterion-based discourse assessment using ecological stimuli, and highlight its potential for clinical implementation.