Dependability of Large Language Models in Cardiovascular Medicine: A Scoping Review.

Abstract

Background: The adoption of large language models (LLMs) in both clinical and consumer healthcare settings has surged exponentially. However, there remains limited evidence on their reliability and impact in cardiovascular practice.

Objectives: This scoping review was designed to consolidate the existing biomedical literature on applicability, reliability, and quality improvement strategies for the integration of LLMs into the cardiovascular domain. Following Cochrane methodology and Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines, three electronic databases (PubMed, Web of Science, and Embase) were systematically searched to identify pertinent studies published between August 2020 and February 2025. Articles addressing the development, implementation, and assessment of LLMs in cardiovascular medicine were selected for comprehensive analysis.

Results: Twenty-five eligible publications evaluated the performance of LLMs in responding to cardiology-related questions, encompassing parameters such as accuracy, response latency, indirectness, completeness, and so on. The assessment methodology varied considerably across studies. LLMs demonstrated potential utility in cardiovascular decision-making, myocarditis management, cardiac arrest diagnosis and treatment, and image differentiation.

Conclusions: Although some LLM-generated responses to cardiovascular-related questions exhibit acceptable levels of quality, significant drawbacks persist. These include verbosity, inaccuracies, occasional misinformation, inconsistent outputs to identical questions, bias, and poor reproducibility. Overall, this work highlights the urgent need for continued refinement and validation.