Ultrasound report generation with fuzzy knowledge and multi-modal large language model

Abstract

Ultrasound report generation is a critical component of computer-aided diagnosis, aimed at alleviating the workload of radiologists during scanning procedures and enhancing diagnostic efficiency. Despite advancements in automatic report generation technologies, the development of a unified framework for generating reports across diverse anatomical regions in ultrasound imaging remains a significant challenge. In this study, we propose a novel and efficient multimodal large language model framework specifically designed for ultrasound report generation. Our framework leverages fuzzy theory to extract essential anatomical knowledge from statistical features, thereby providing more accurate and context-aware guidance throughout the report generation process. Furthermore, we propose a novel evaluation metric designed to assess both the precision and the clinical significance of the generated reports, leveraging insights derived from deep domain expertise. In contrast to traditional evaluation methods, this metric offers a more comprehensive and clinically meaningful assessment. To validate the efficacy of our framework, we conduct extensive experiments on both a publicly available dataset and a proprietary dataset collected from the First Affiliated Hospital of Sun Yat-sen University. We also supplemented our proprietary ultrasound dataset with an external validation set collected from Foshan Sanshui Hospital and The First Affiliated Hospital of Guangzhou. Experimental results demonstrate that our approach consistently achieves state-of-the-art performance across multiple evaluation metrics, highlighting its robustness and adaptability. These findings underscore the potential of our framework in advancing the accuracy and clinical applicability of ultrasound report generation.