Benchmarking and fine-tuning vision-language models on a visual question answering dataset for myopic maculopathy

IF 7 3区医学 Q1 OPHTHALMOLOGY

Asia-Pacific Journal of Ophthalmology Pub Date : 2026-07-01 Epub Date: 2026-06-26 DOI:10.1016/j.apjo.2026.100345

Tsun Hei Yip, Pusheng Xu, Zirong Liu, Ebenezer Zaabaar, Angie H.C. Fong, Yatu Guo, Ziming Xu, Wei Zhang, Yau Kei Chan, Dennis S.C. Lam, Kendrick C. Shih, Christopher K.S. Leung, Xiu Juan Zhang

{"title":"Benchmarking and fine-tuning vision-language models on a visual question answering dataset for myopic maculopathy","authors":"Tsun Hei Yip, Pusheng Xu, Zirong Liu, Ebenezer Zaabaar, Angie H.C. Fong, Yatu Guo, Ziming Xu, Wei Zhang, Yau Kei Chan, Dennis S.C. Lam, Kendrick C. Shih, Christopher K.S. Leung, Xiu Juan Zhang","doi":"10.1016/j.apjo.2026.100345","DOIUrl":null,"url":null,"abstract":"<div><h3>Purpose</h3><div>To build a visual question answering (VQA) dataset for fine-tuning and evaluating vision-language models (VLMs) in myopic maculopathy (MM).</div></div><div><h3>Design</h3><div>Cross-sectional study.</div></div><div><h3>Methods</h3><div>Colour fundus photographs (CFPs) from two publicly available datasets were graded using META-PM classification system. GPT–5 was used to generate clinical captions, true/false [TFQ] and open-ended [OEQ] question-answer pairs, all of which were manually verified. InternVL3‐8B was fine-tuned on this dataset and evaluated against Gemini 3 Pro, Claude Sonnet 4.5, Qwen3-VL-30B-A3B-Instruct, and pre-trained InternVL3‐8B. OEQ responses was evaluated by GPT–5 using a three-level scoring system (0, completely incorrect; 0.5, partially correct; 1, fully correct) and summarized as weighted accuracy. Overall accuracy was defined as the arithmetic mean of the TFQ and OEQ accuracies.</div></div><div><h3>Results</h3><div>MM-VQA comprises 2591 CFPs and 19,648 question-answer pairs. Fine-tuned InternVL3-8B model achieved an overall accuracy of 0.746, surpassing Claude Sonnet 4.5 (0.596), Qwen3-VL-30B-A3B-Instruct (0.566), and pre-trained InternVL3-8B (0.428) (all <em>P</em> < 0.001), while showing no significant difference compared with Gemini 3 Pro (0.724, <em>P</em> = 0.642). For TFQ, the fine-tuned model reached an accuracy of 0.919, outperforming Gemini 3 Pro (0.881), Qwen3-VL-30B-A3B-Instruct (0.834), Claude Sonnet 4.5 (0.796), and the pre-trained model (0.696) (all <em>P</em> < 0.001). On OEQ, it also ranked highest (0.572), outperforming Gemini 3 Pro (0.567, <em>P</em> = 0.044), Claude Sonnet 4.5 (0.395, <em>P</em> < 0.001), Qwen3-VL-30B-A3B-Instruct (0.297, <em>P</em> < 0.001) and the pre-trained model (0.160, <em>P</em> < 0.001).</div></div><div><h3>Conclusion</h3><div>This study provides a valuable VQA dataset for MM, supporting the development of disease-specialised VLMs in ophthalmology.</div></div>","PeriodicalId":8594,"journal":{"name":"Asia-Pacific Journal of Ophthalmology","volume":"15 4","pages":"Article 100345"},"PeriodicalIF":7.0000,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Asia-Pacific Journal of Ophthalmology","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S216209892600071X","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2026/6/26 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"OPHTHALMOLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose

To build a visual question answering (VQA) dataset for fine-tuning and evaluating vision-language models (VLMs) in myopic maculopathy (MM).

Design

Cross-sectional study.

Methods

Colour fundus photographs (CFPs) from two publicly available datasets were graded using META-PM classification system. GPT–5 was used to generate clinical captions, true/false [TFQ] and open-ended [OEQ] question-answer pairs, all of which were manually verified. InternVL3‐8B was fine-tuned on this dataset and evaluated against Gemini 3 Pro, Claude Sonnet 4.5, Qwen3-VL-30B-A3B-Instruct, and pre-trained InternVL3‐8B. OEQ responses was evaluated by GPT–5 using a three-level scoring system (0, completely incorrect; 0.5, partially correct; 1, fully correct) and summarized as weighted accuracy. Overall accuracy was defined as the arithmetic mean of the TFQ and OEQ accuracies.

Results

MM-VQA comprises 2591 CFPs and 19,648 question-answer pairs. Fine-tuned InternVL3-8B model achieved an overall accuracy of 0.746, surpassing Claude Sonnet 4.5 (0.596), Qwen3-VL-30B-A3B-Instruct (0.566), and pre-trained InternVL3-8B (0.428) (all P < 0.001), while showing no significant difference compared with Gemini 3 Pro (0.724, P = 0.642). For TFQ, the fine-tuned model reached an accuracy of 0.919, outperforming Gemini 3 Pro (0.881), Qwen3-VL-30B-A3B-Instruct (0.834), Claude Sonnet 4.5 (0.796), and the pre-trained model (0.696) (all P < 0.001). On OEQ, it also ranked highest (0.572), outperforming Gemini 3 Pro (0.567, P = 0.044), Claude Sonnet 4.5 (0.395, P < 0.001), Qwen3-VL-30B-A3B-Instruct (0.297, P < 0.001) and the pre-trained model (0.160, P < 0.001).

Conclusion

This study provides a valuable VQA dataset for MM, supporting the development of disease-specialised VLMs in ophthalmology.

查看原文本刊更多论文

对近视黄斑病变视觉问答数据集上的视觉语言模型进行基准测试和微调。

目的：建立一个视觉问答（VQA）数据集，用于对近视黄斑病变（MM）的视觉语言模型（VLMs）进行微调和评估。设计：横断面研究。方法：使用META-PM分类系统对来自两个公开数据集的彩色眼底照片（CFPs）进行分级。使用GPT-5生成临床字幕、真假[TFQ]和开放式[OEQ]问答对，并进行人工验证。InternVL3-8B在此数据集上进行了微调，并与Gemini 3 Pro、Claude Sonnet 4.5、qwen3 - v1 - 30b - a3b - instruct和预训练的InternVL3-8B进行了评估。通过GPT-5对OEQ反应进行评估，采用三级评分系统（0，完全不正确；0.5，部分正确；1，完全正确），并总结为加权准确性。总体精度定义为TFQ和OEQ精度的算术平均值。结果：MM-VQA包括2591个cfp和19648个问答对。微调后的InternVL3-8B模型总体准确率为0.746，超过Claude Sonnet 4.5（0.596）、qwen3 - v1 - 30b - a3b - instruct（0.566）和预训练后的InternVL3-8B (0.428) (P均< 0.001)，与Gemini 3 Pro （0.724, P = 0.642）相比差异无统计学意义。对于TFQ，微调模型的准确率为0.919，优于Gemini 3 Pro（0.881）、Qwen3-VL-30B-A3B-Instruct（0.834）、Claude Sonnet 4.5（0.796）和预训练模型（0.696）（均P < 0.001）。在OEQ方面，它也排名最高（0.572），优于Gemini 3 Pro （0.567, P = 0.044）、Claude Sonnet 4.5 （0.395, P < 0.001）、Qwen3-VL-30B-A3B-Instruct （0.297, P < 0.001）和预训练模型（0.160,P < 0.001）。结论：本研究为MM提供了一个有价值的VQA数据集，支持眼科疾病特异性VLMs的发展。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Asia-Pacific Journal of Ophthalmology OPHTHALMOLOGY-

CiteScore

8.10

自引率

18.20%

发文量

197

审稿时长

6 weeks

期刊介绍： The Asia-Pacific Journal of Ophthalmology, a bimonthly, peer-reviewed online scientific publication, is an official publication of the Asia-Pacific Academy of Ophthalmology (APAO), a supranational organization which is committed to research, training, learning, publication and knowledge and skill transfers in ophthalmology and visual sciences. The Asia-Pacific Journal of Ophthalmology welcomes review articles on currently hot topics, original, previously unpublished manuscripts describing clinical investigations, clinical observations and clinically relevant laboratory investigations, as well as .perspectives containing personal viewpoints on topics with broad interests. Editorials are published by invitation only. Case reports are generally not considered. The Asia-Pacific Journal of Ophthalmology covers 16 subspecialties and is freely circulated among individual members of the APAO’s member societies, which amounts to a potential readership of over 50,000.