MedVH: Toward Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Context.

Abstract

Large vision language models (LVLMs) have achieved superior performance on natural image and text tasks, inspiring extensive fine-tuning research. However, their robustness against hallucination in clinical contexts remains understudied. We propose the Medical Visual Hallucination Test (MedVH), a novel evaluation framework assessing hallucination tendencies in both medical-specific and general-purpose LVLMs. MedVH encompasses six tasks targeting medical hallucinations, including two traditional tasks and four novel tasks formatted as multi-choice visual question answering and long response generation. Our extensive experiments with six evaluation metrics reveal that medical LVLMs, despite promising performance on standard medical tasks, are particularly susceptible to hallucinations-often more so than general models. This raises significant concerns about domain-specific model reliability. For real-world applications, medical LVLMs must accurately integrate medical knowledge while maintaining robust reasoning to prevent hallucination. We explore mitigation methods without model-specific fine-tuning, including prompt engineering and collaboration between general and domain-specific models. Our work provides a foundation for future evaluation studies. The dataset is available at PhysioNet: https://physionet.org/content/medvh.