An agentic AI system for multi-framework communication coding

Abstract

Abstract Clinical communication is central to patient outcomes, yet large-scale annotation of patient–provider conversations remains labor-intensive, inconsistent, and difficult to scale. Existing large language model (LLM)–based approaches typically rely on single-task systems with limited robustness across communication frameworks and clinical domains. In this study, we developed MOSAIC, a M ulti-framew O rk S tructured Agentic AI system for scalable Clinical C ommunication coding. MOSAIC combines dynamically few-shot prompting, retrieval-augmented generation (RAG), and a LangGraph-based multi-agent workflow for planning, annotation, and verification. Performance was evaluated against gold-standard annotations produced by trained human coders. MOSAIC was developed on 54 annotated transcripts and evaluated on 120 transcripts spanning rheumatology and obstetrics/gynecology (OB/GYN). On the evaluation set, MOSAIC achieved an overall weighted F1 score of 95.6%. Performance was highest in rheumatology (F1 = 95.9%) and strongest for bias and patient behaviors categories (F1 = 96.4% and 96.1%, respectively). These results demonstrate the potential of multi-framework AI systems for reliable clinical communication analysis.