Visual-language foundation models in medical imaging: A systematic review and meta-analysis of diagnostic and analytical applications

Abstract

Background and objective

Visual-language foundation models (VLMs) have garnered attention for their numerous advantages and significant potential in AI-aided diagnosis and treatment, driving widespread applications in medical tasks. This study analyzes and summarizes the value and prospects of VLMs, highlighting their groundbreaking opportunities in healthcare.

Methods

This systematic review and meta-analysis, registered with PROSPERO (CRD42024575746), included studies from PubMed, Embase, Web of Science, and IEEE from inception to December 31, 2024. The inclusion criteria covered state-of-the-art VLM developments and applications in medical imaging. Metrics such as AUC, Dice coefficient, BLEU score, and Accuracy were pooled for tasks like classification, segmentation, report generation, and Visual Question Answering (VQA). Reporting quality and bias were assessed using the QUADAS-AI checklist.

Results

A total of 106 eligible studies were identified for this systematic review, of which 94 were included for meta-analysis. The pooled AUC for downstream classification tasks was 0.86 (0.85–0.87); pooled Dice coefficient for segmentation tasks was 0.73 (0.68–0.78); pooled BLEU score for report generation tasks was 0.31 (0.20–0.43); and pooled Acc score for VQA was 0.76 (0.71–0.81). Subgroup analyses were stratified by imaging modalities (radiological, pathological and surface imaging) and publication year (before or after 2023) to explore the heterogeneity within VLM research and to analyze diagnostic performance of the VLMs under different conditions.

Conclusions

VLMs based on medical imaging have demonstrated strong performance and significant potential in computer-assisted clinical diagnosis. Stricter reporting standards addressing the unique challenges of VLM research could enhance study quality.