Retrieval-to-abstraction LLM-assisted literature synthesis: Case demonstrations in digital health technology-enabled randomized controlled trial outcome reporting.

Abstract

Systematic reviews of randomized controlled trials (RCTs) support evidence-based clinical advancement. Clinical trial reporting guidelines, including CONSORT, establish common data elements that support systematic synthesis of published trial outcomes. However, the rapid rise of digital health technologies presents new challenges for reporting guidelines, and substantial real-world reporting heterogeneity has been observed. Therefore, machine-assisted evaluation of reporting adherence offers a pathway for more agile information management to accommodate the diverse and disruptive nature of digital health technology-enabled RCTs. Here, we propose an accelerated retrieval-to-abstraction literature synthesis framework to conduct article screening, text and table data preprocessing, and full-text review using an unmodified pretrained large language model. We apply this methodology in two digital health technology-enabled RCT case studies evaluating adherence to technology and digital health equity guideline reporting items. Large language model performance was on par with inter-annotator agreement F1 scores (≥0.9) in both screening and information-extraction tasks. These findings support the scalability and rigor of accelerated evidence synthesis and demonstrate the importance of common reporting elements across digital health technology-enabled RCTs.