Multimodal large language models address clinical queries in laryngeal cancer surgery: a comparative evaluation of image interpretation across different models.

Abstract

Background and objectives: Recent advances in multimodal large language models (MLLMs) have shown promise in medical image interpretation, yet their utility in surgical contexts remains unexplored. This study evaluates six MLLMs' performance in interpreting diverse imaging modalities for laryngeal cancer surgery.

Methods: We analyzed 169 images (X-rays, CT scans, laryngoscopy, and pathology findings) from 50 patients using six state-of-the-art MLLMs. Model performance was assessed across 1084 clinically relevant questions by two independent physicians.

Results: Claude 3.5 Sonnet achieves the highest accuracy (79.43%, 95% CI: 77.02%-81.84%). Performance varied significantly across imaging modalities and between commercial and open-source models, with a 19-percentage point gap between the best commercial and open-source solutions.

Conclusion: Advanced MLLMs show promising potential as clinical decision support tools in laryngeal cancer surgery, while performance variations suggest the need for specialized model development and clinical workflow integration. Future research should focus on developing specialized MLLMs trained on large-scale multi-center laryngeal cancer datasets.