Automating Adjudication of Cardiovascular Events Using Large Language Models.

Abstract

Cardiovascular event adjudication is essential in clinical trials but relies on manual chart review that is slow, variable, and expensive. We present a two-stage framework that automates adjudication of cardiovascular deaths using large language models (LLMs). First, a few-shot LLM extracts structured evidence (event, span, negation, date) from unstructured clinical documents. Second, a Tree-of-Thoughts adjudicator aligns its reasoning with clinical endpoint committee (CEC) guidelines to classify deaths as cardiovascular or non-cardiovascular and produce an auditable rationale. On Lilly clinical-trial data, extraction achieved precision 0.96, recall 0.71 (F1 0.82), and adjudication attained 0.68 accuracy (GPT-4 ToT), outperforming a summarizer-plus-adjudicator baseline. We introduce CLEART, a rubric-based automated score that quantifies rationale quality across clarity, consistency, detail, guideline adherence, relevance, and timeline accuracy (overall 0.67), highlighting temporal reasoning and relevance as key areas for improvement. This approach can reduce adjudication time and variability while increasing transparency.