一种新的基于基础模型的多模态视网膜年龄预测框架

IF 3.7 3区 医学 Q2 ENGINEERING, BIOMEDICAL
Christopher Nielsen;Matthias Wilms;Nils D. Forkert
{"title":"一种新的基于基础模型的多模态视网膜年龄预测框架","authors":"Christopher Nielsen;Matthias Wilms;Nils D. Forkert","doi":"10.1109/JTEHM.2025.3576596","DOIUrl":null,"url":null,"abstract":"The retinal age gap (RAG; the difference between the retina’s biological and chronological age) has recently gained increased attention as a potential image-based, non-invasive, and accessible biomarker for a broad spectrum of ocular and non-ocular diseases. Traditionally, machine learning predictions of biological retinal age utilize convolutional neural network (CNN) architectures and data from color fundus photography (CFP). Despite being previously unexplored, the multimodal fusion of two-dimensional CFP with three-dimensional optical coherence tomography (OCT) data has significant potential to enhance retinal age prediction accuracy and the diagnostic utility of the RAG biomarker. Therefore, this work presents a novel foundation model-based framework for multimodal retinal age prediction. Technology or Method: Feature representations from CFP and OCT images were extracted using RETFound, a powerful foundation model for retinal image analysis. These representations were then combined using an innovative fusion strategy to train a lightweight linear regression head model for predicting retinal age. Training and evaluation of the developed multimodal retinal age prediction model was achieved using retinal images from over 80,000 participants in the UK Biobank. Results: The developed multimodal model sets a new benchmark in retinal age prediction (mean absolute error of 2.75 years), outperforming traditional CNN and single-modality approaches. Additionally, multimodal RAG values demonstrated superior performance in classifying patients with diabetes mellitus type 1, multiple sclerosis, and chronic kidney disease, highlighting the clinical relevance of the proposed multimodal approach for non-ocular disease detection. Conclusions: This work demonstrates that multimodal fusion of CFP and OCT significantly improves retinal age prediction and subsequent RAG-based analyses. By leveraging foundation models and multimodal retinal imaging, the proposed approach enhances disease classification accuracy and demonstrates the potential of integrating the RAG into clinical workflows as a scalable, non-invasive screening tool. Significance: The findings underscore the potential of multimodal retinal imaging to transform RAG into a clinically relevant and highly accessible biomarker for disease detection.","PeriodicalId":54255,"journal":{"name":"IEEE Journal of Translational Engineering in Health and Medicine-Jtehm","volume":"13 ","pages":"299-309"},"PeriodicalIF":3.7000,"publicationDate":"2025-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11023594","citationCount":"0","resultStr":"{\"title\":\"A Novel Foundation Model-Based Framework for Multimodal Retinal Age Prediction\",\"authors\":\"Christopher Nielsen;Matthias Wilms;Nils D. Forkert\",\"doi\":\"10.1109/JTEHM.2025.3576596\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The retinal age gap (RAG; the difference between the retina’s biological and chronological age) has recently gained increased attention as a potential image-based, non-invasive, and accessible biomarker for a broad spectrum of ocular and non-ocular diseases. Traditionally, machine learning predictions of biological retinal age utilize convolutional neural network (CNN) architectures and data from color fundus photography (CFP). Despite being previously unexplored, the multimodal fusion of two-dimensional CFP with three-dimensional optical coherence tomography (OCT) data has significant potential to enhance retinal age prediction accuracy and the diagnostic utility of the RAG biomarker. Therefore, this work presents a novel foundation model-based framework for multimodal retinal age prediction. Technology or Method: Feature representations from CFP and OCT images were extracted using RETFound, a powerful foundation model for retinal image analysis. These representations were then combined using an innovative fusion strategy to train a lightweight linear regression head model for predicting retinal age. Training and evaluation of the developed multimodal retinal age prediction model was achieved using retinal images from over 80,000 participants in the UK Biobank. Results: The developed multimodal model sets a new benchmark in retinal age prediction (mean absolute error of 2.75 years), outperforming traditional CNN and single-modality approaches. Additionally, multimodal RAG values demonstrated superior performance in classifying patients with diabetes mellitus type 1, multiple sclerosis, and chronic kidney disease, highlighting the clinical relevance of the proposed multimodal approach for non-ocular disease detection. Conclusions: This work demonstrates that multimodal fusion of CFP and OCT significantly improves retinal age prediction and subsequent RAG-based analyses. By leveraging foundation models and multimodal retinal imaging, the proposed approach enhances disease classification accuracy and demonstrates the potential of integrating the RAG into clinical workflows as a scalable, non-invasive screening tool. Significance: The findings underscore the potential of multimodal retinal imaging to transform RAG into a clinically relevant and highly accessible biomarker for disease detection.\",\"PeriodicalId\":54255,\"journal\":{\"name\":\"IEEE Journal of Translational Engineering in Health and Medicine-Jtehm\",\"volume\":\"13 \",\"pages\":\"299-309\"},\"PeriodicalIF\":3.7000,\"publicationDate\":\"2025-06-04\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11023594\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Journal of Translational Engineering in Health and Medicine-Jtehm\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/11023594/\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ENGINEERING, BIOMEDICAL\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Journal of Translational Engineering in Health and Medicine-Jtehm","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/11023594/","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, BIOMEDICAL","Score":null,"Total":0}
引用次数: 0

摘要

视网膜年龄差距(RAG;视网膜的生物年龄和实足年龄之间的差异最近作为一种潜在的基于图像的、非侵入性的、可获得的生物标志物,广泛用于眼部和非眼部疾病,得到了越来越多的关注。传统上,生物视网膜年龄的机器学习预测利用卷积神经网络(CNN)架构和彩色眼底摄影(CFP)数据。尽管以前未被探索过,但二维CFP与三维光学相干断层扫描(OCT)数据的多模态融合具有显著的潜力,可以提高视网膜年龄预测的准确性和RAG生物标志物的诊断效用。因此,这项工作提出了一种新的基于基础模型的多模态视网膜年龄预测框架。技术或方法:使用RETFound(视网膜图像分析的强大基础模型)提取CFP和OCT图像的特征表示。然后使用一种创新的融合策略将这些表示组合起来,以训练用于预测视网膜年龄的轻量级线性回归头部模型。利用来自英国生物银行80,000多名参与者的视网膜图像,对开发的多模态视网膜年龄预测模型进行了训练和评估。结果:所建立的多模态模型为视网膜年龄预测设定了新的基准(平均绝对误差为2.75年),优于传统的CNN和单模态方法。此外,多模态RAG值在对1型糖尿病、多发性硬化症和慢性肾病患者进行分类方面表现出优异的性能,突出了所提出的多模态方法在非眼部疾病检测中的临床意义。结论:本研究表明,CFP和OCT的多模态融合显著改善了视网膜年龄预测和随后基于rag的分析。通过利用基础模型和多模态视网膜成像,所提出的方法提高了疾病分类的准确性,并展示了将RAG作为可扩展的非侵入性筛查工具整合到临床工作流程中的潜力。意义:研究结果强调了多模态视网膜成像将RAG转化为临床相关且高度可及的疾病检测生物标志物的潜力。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
A Novel Foundation Model-Based Framework for Multimodal Retinal Age Prediction
The retinal age gap (RAG; the difference between the retina’s biological and chronological age) has recently gained increased attention as a potential image-based, non-invasive, and accessible biomarker for a broad spectrum of ocular and non-ocular diseases. Traditionally, machine learning predictions of biological retinal age utilize convolutional neural network (CNN) architectures and data from color fundus photography (CFP). Despite being previously unexplored, the multimodal fusion of two-dimensional CFP with three-dimensional optical coherence tomography (OCT) data has significant potential to enhance retinal age prediction accuracy and the diagnostic utility of the RAG biomarker. Therefore, this work presents a novel foundation model-based framework for multimodal retinal age prediction. Technology or Method: Feature representations from CFP and OCT images were extracted using RETFound, a powerful foundation model for retinal image analysis. These representations were then combined using an innovative fusion strategy to train a lightweight linear regression head model for predicting retinal age. Training and evaluation of the developed multimodal retinal age prediction model was achieved using retinal images from over 80,000 participants in the UK Biobank. Results: The developed multimodal model sets a new benchmark in retinal age prediction (mean absolute error of 2.75 years), outperforming traditional CNN and single-modality approaches. Additionally, multimodal RAG values demonstrated superior performance in classifying patients with diabetes mellitus type 1, multiple sclerosis, and chronic kidney disease, highlighting the clinical relevance of the proposed multimodal approach for non-ocular disease detection. Conclusions: This work demonstrates that multimodal fusion of CFP and OCT significantly improves retinal age prediction and subsequent RAG-based analyses. By leveraging foundation models and multimodal retinal imaging, the proposed approach enhances disease classification accuracy and demonstrates the potential of integrating the RAG into clinical workflows as a scalable, non-invasive screening tool. Significance: The findings underscore the potential of multimodal retinal imaging to transform RAG into a clinically relevant and highly accessible biomarker for disease detection.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
CiteScore
7.40
自引率
2.90%
发文量
65
审稿时长
27 weeks
期刊介绍: The IEEE Journal of Translational Engineering in Health and Medicine is an open access product that bridges the engineering and clinical worlds, focusing on detailed descriptions of advanced technical solutions to a clinical need along with clinical results and healthcare relevance. The journal provides a platform for state-of-the-art technology directions in the interdisciplinary field of biomedical engineering, embracing engineering, life sciences and medicine. A unique aspect of the journal is its ability to foster a collaboration between physicians and engineers for presenting broad and compelling real world technological and engineering solutions that can be implemented in the interest of improving quality of patient care and treatment outcomes, thereby reducing costs and improving efficiency. The journal provides an active forum for clinical research and relevant state-of the-art technology for members of all the IEEE societies that have an interest in biomedical engineering as well as reaching out directly to physicians and the medical community through the American Medical Association (AMA) and other clinical societies. The scope of the journal includes, but is not limited, to topics on: Medical devices, healthcare delivery systems, global healthcare initiatives, and ICT based services; Technological relevance to healthcare cost reduction; Technology affecting healthcare management, decision-making, and policy; Advanced technical work that is applied to solving specific clinical needs.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信