Theodore Spaide PhD , Anand E. Rajesh MD , Nayoon Gim , Marian Blazes MD , Cecilia S. Lee MD, MS , Niranchana Macivannan PhD , Gary Lee PhD, MEng , Warren Lewis MS , Ali Salehi PhD , Luis de Sisternes PhD , Gissel Herrera MD , Mengxi Shen MD, PhD , Giovanni Gregori PhD , Philip J. Rosenfeld MD, PhD , Varsha Pramil MD, MS , Nadia Waheed MD, MPH , Yue Wu PhD , Qinqin Zhang PhD , Aaron Y. Lee MD, MSCI
{"title":"利用贝叶斯深度学习估计地理萎缩分割的不确定性","authors":"Theodore Spaide PhD , Anand E. Rajesh MD , Nayoon Gim , Marian Blazes MD , Cecilia S. Lee MD, MS , Niranchana Macivannan PhD , Gary Lee PhD, MEng , Warren Lewis MS , Ali Salehi PhD , Luis de Sisternes PhD , Gissel Herrera MD , Mengxi Shen MD, PhD , Giovanni Gregori PhD , Philip J. Rosenfeld MD, PhD , Varsha Pramil MD, MS , Nadia Waheed MD, MPH , Yue Wu PhD , Qinqin Zhang PhD , Aaron Y. Lee MD, MSCI","doi":"10.1016/j.xops.2024.100587","DOIUrl":null,"url":null,"abstract":"<div><h3>Purpose</h3><p>To apply methods for quantifying uncertainty of deep learning segmentation of geographic atrophy (GA).</p></div><div><h3>Design</h3><p>Retrospective analysis of OCT images and model comparison.</p></div><div><h3>Participants</h3><p>One hundred twenty-six eyes from 87 participants with GA in the SWAGGER cohort of the Nonexudative Age-Related Macular Degeneration Imaged with Swept-Source OCT (SS-OCT) study.</p></div><div><h3>Methods</h3><p>The manual segmentations of GA lesions were conducted on structural subretinal pigment epithelium en face images from the SS-OCT images. Models were developed for 2 approximate Bayesian deep learning techniques, Monte Carlo dropout and ensemble, to assess the uncertainty of GA semantic segmentation and compared to a traditional deep learning model.</p></div><div><h3>Main Outcome Measures</h3><p>Model performance (Dice score) was compared. Uncertainty was calculated using the formula for Shannon Entropy.</p></div><div><h3>Results</h3><p>The output of both Bayesian technique models showed a greater number of pixels with high entropy than the standard model. Dice scores for the Monte Carlo dropout method (0.90, 95% confidence interval 0.87–0.93) and the ensemble method (0.88, 95% confidence interval 0.85–0.91) were significantly higher (<em>P</em> < 0.001) than for the traditional model (0.82, 95% confidence interval 0.78–0.86).</p></div><div><h3>Conclusions</h3><p>Quantifying the uncertainty in a prediction of GA may improve trustworthiness of the models and aid clinicians in decision-making. The Bayesian deep learning techniques generated pixel-wise estimates of model uncertainty for segmentation, while also improving model performance compared with traditionally trained deep learning models.</p></div><div><h3>Financial Disclosures</h3><p>Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.</p></div>","PeriodicalId":74363,"journal":{"name":"Ophthalmology science","volume":"5 1","pages":"Article 100587"},"PeriodicalIF":3.2000,"publicationDate":"2024-07-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S2666914524001234/pdfft?md5=678a9a10974ce3ddf09356f4abea5102&pid=1-s2.0-S2666914524001234-main.pdf","citationCount":"0","resultStr":"{\"title\":\"Estimating Uncertainty of Geographic Atrophy Segmentations with Bayesian Deep Learning\",\"authors\":\"Theodore Spaide PhD , Anand E. Rajesh MD , Nayoon Gim , Marian Blazes MD , Cecilia S. Lee MD, MS , Niranchana Macivannan PhD , Gary Lee PhD, MEng , Warren Lewis MS , Ali Salehi PhD , Luis de Sisternes PhD , Gissel Herrera MD , Mengxi Shen MD, PhD , Giovanni Gregori PhD , Philip J. Rosenfeld MD, PhD , Varsha Pramil MD, MS , Nadia Waheed MD, MPH , Yue Wu PhD , Qinqin Zhang PhD , Aaron Y. Lee MD, MSCI\",\"doi\":\"10.1016/j.xops.2024.100587\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><h3>Purpose</h3><p>To apply methods for quantifying uncertainty of deep learning segmentation of geographic atrophy (GA).</p></div><div><h3>Design</h3><p>Retrospective analysis of OCT images and model comparison.</p></div><div><h3>Participants</h3><p>One hundred twenty-six eyes from 87 participants with GA in the SWAGGER cohort of the Nonexudative Age-Related Macular Degeneration Imaged with Swept-Source OCT (SS-OCT) study.</p></div><div><h3>Methods</h3><p>The manual segmentations of GA lesions were conducted on structural subretinal pigment epithelium en face images from the SS-OCT images. Models were developed for 2 approximate Bayesian deep learning techniques, Monte Carlo dropout and ensemble, to assess the uncertainty of GA semantic segmentation and compared to a traditional deep learning model.</p></div><div><h3>Main Outcome Measures</h3><p>Model performance (Dice score) was compared. Uncertainty was calculated using the formula for Shannon Entropy.</p></div><div><h3>Results</h3><p>The output of both Bayesian technique models showed a greater number of pixels with high entropy than the standard model. Dice scores for the Monte Carlo dropout method (0.90, 95% confidence interval 0.87–0.93) and the ensemble method (0.88, 95% confidence interval 0.85–0.91) were significantly higher (<em>P</em> < 0.001) than for the traditional model (0.82, 95% confidence interval 0.78–0.86).</p></div><div><h3>Conclusions</h3><p>Quantifying the uncertainty in a prediction of GA may improve trustworthiness of the models and aid clinicians in decision-making. The Bayesian deep learning techniques generated pixel-wise estimates of model uncertainty for segmentation, while also improving model performance compared with traditionally trained deep learning models.</p></div><div><h3>Financial Disclosures</h3><p>Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.</p></div>\",\"PeriodicalId\":74363,\"journal\":{\"name\":\"Ophthalmology science\",\"volume\":\"5 1\",\"pages\":\"Article 100587\"},\"PeriodicalIF\":3.2000,\"publicationDate\":\"2024-07-24\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.sciencedirect.com/science/article/pii/S2666914524001234/pdfft?md5=678a9a10974ce3ddf09356f4abea5102&pid=1-s2.0-S2666914524001234-main.pdf\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Ophthalmology science\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S2666914524001234\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"OPHTHALMOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Ophthalmology science","FirstCategoryId":"1085","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2666914524001234","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"OPHTHALMOLOGY","Score":null,"Total":0}
引用次数: 0
摘要
目的应用各种方法量化深度学习分割地理萎缩(GA)的不确定性.设计对OCT图像进行回顾性分析并进行模型比较.参与者SWAGGER队列中87名患有GA的参与者的126只眼睛.方法在SS-OCT图像的结构性视网膜下色素上皮表面图像上对GA病变进行人工分割.为评估GA语义分割的不确定性,开发了2种近似贝叶斯深度学习技术(蒙特卡洛剔除和集合)模型,并与传统深度学习模型进行了比较。结果两种贝叶斯技术模型的输出都比标准模型显示出更多的高熵像素。蒙特卡洛剔除法(0.90,95% 置信区间 0.87-0.93)和集合法(0.88,95% 置信区间 0.85-0.91)的骰子得分显著高于传统模型(0.82,95% 置信区间 0.78-0.86)(P <0.001)。结论量化 GA 预测中的不确定性可提高模型的可信度,帮助临床医生做出决策。与传统训练的深度学习模型相比,贝叶斯深度学习技术能对模型的不确定性进行像素级估计,同时还能提高模型的性能。
Estimating Uncertainty of Geographic Atrophy Segmentations with Bayesian Deep Learning
Purpose
To apply methods for quantifying uncertainty of deep learning segmentation of geographic atrophy (GA).
Design
Retrospective analysis of OCT images and model comparison.
Participants
One hundred twenty-six eyes from 87 participants with GA in the SWAGGER cohort of the Nonexudative Age-Related Macular Degeneration Imaged with Swept-Source OCT (SS-OCT) study.
Methods
The manual segmentations of GA lesions were conducted on structural subretinal pigment epithelium en face images from the SS-OCT images. Models were developed for 2 approximate Bayesian deep learning techniques, Monte Carlo dropout and ensemble, to assess the uncertainty of GA semantic segmentation and compared to a traditional deep learning model.
Main Outcome Measures
Model performance (Dice score) was compared. Uncertainty was calculated using the formula for Shannon Entropy.
Results
The output of both Bayesian technique models showed a greater number of pixels with high entropy than the standard model. Dice scores for the Monte Carlo dropout method (0.90, 95% confidence interval 0.87–0.93) and the ensemble method (0.88, 95% confidence interval 0.85–0.91) were significantly higher (P < 0.001) than for the traditional model (0.82, 95% confidence interval 0.78–0.86).
Conclusions
Quantifying the uncertainty in a prediction of GA may improve trustworthiness of the models and aid clinicians in decision-making. The Bayesian deep learning techniques generated pixel-wise estimates of model uncertainty for segmentation, while also improving model performance compared with traditionally trained deep learning models.
Financial Disclosures
Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.