Rethinking data imbalance in class incremental surgical instrument segmentation

IF 11.8 1区 医学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE
Shifang Zhao , Long Bai , Kun Yuan , Feng Li , Jieming Yu , Wenzhen Dong , Guankun Wang , Mobarakol Islam , Nicolas Padoy , Nassir Navab , Hongliang Ren
{"title":"Rethinking data imbalance in class incremental surgical instrument segmentation","authors":"Shifang Zhao ,&nbsp;Long Bai ,&nbsp;Kun Yuan ,&nbsp;Feng Li ,&nbsp;Jieming Yu ,&nbsp;Wenzhen Dong ,&nbsp;Guankun Wang ,&nbsp;Mobarakol Islam ,&nbsp;Nicolas Padoy ,&nbsp;Nassir Navab ,&nbsp;Hongliang Ren","doi":"10.1016/j.media.2025.103728","DOIUrl":null,"url":null,"abstract":"<div><div>In surgical instrument segmentation, the increasing variety of instruments over time poses a significant challenge for existing neural networks, as they are unable to effectively learn such incremental tasks and suffer from catastrophic forgetting. When learning new data, the model experiences a sharp performance drop on previously learned data. Although several continual learning methods have been proposed for incremental understanding tasks in surgical scenarios, the issue of data imbalance often leads to a strong bias in the segmentation head, resulting in poor performance. Data imbalance can occur in two forms: (i) class imbalance between new and old data, and (ii) class imbalance within the same time point of data. Such imbalances often cause the dominant classes to take over the training process of continual semantic segmentation (CSS). To address this issue, we propose <strong>SurgCSS</strong>, a novel plug-and-play CSS framework for surgical instrument segmentation under data imbalance. Specifically, we generate realistic surgical backgrounds through inpainting and blend instrument foregrounds with the generated backgrounds in a class-aware manner to balance the data distribution in various scenarios. We further propose the Class Desensitization Loss by employing contrastive learning to correct edge biases caused by data imbalance. Moreover, we dynamically fuse the weight parameters of the old and new models to achieve a better trade-off between the biased and unbiased model weights. To investigate the data imbalance problem in surgical scenarios, we construct a new benchmark for surgical instrument CSS by integrating four public datasets: EndoVis 2017, EndoVis 2018, CholecSeg8k, and SAR-RAPR50. Extensive experiments demonstrate the effectiveness of the proposed framework, achieving significant performance improvement against existing baselines. Our method demonstrates excellent potential for clinical applications. The code is publicly available at <span><span>github.com/Zzsf11/SurgCSS</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":18328,"journal":{"name":"Medical image analysis","volume":"105 ","pages":"Article 103728"},"PeriodicalIF":11.8000,"publicationDate":"2025-07-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Medical image analysis","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1361841525002750","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0

Abstract

In surgical instrument segmentation, the increasing variety of instruments over time poses a significant challenge for existing neural networks, as they are unable to effectively learn such incremental tasks and suffer from catastrophic forgetting. When learning new data, the model experiences a sharp performance drop on previously learned data. Although several continual learning methods have been proposed for incremental understanding tasks in surgical scenarios, the issue of data imbalance often leads to a strong bias in the segmentation head, resulting in poor performance. Data imbalance can occur in two forms: (i) class imbalance between new and old data, and (ii) class imbalance within the same time point of data. Such imbalances often cause the dominant classes to take over the training process of continual semantic segmentation (CSS). To address this issue, we propose SurgCSS, a novel plug-and-play CSS framework for surgical instrument segmentation under data imbalance. Specifically, we generate realistic surgical backgrounds through inpainting and blend instrument foregrounds with the generated backgrounds in a class-aware manner to balance the data distribution in various scenarios. We further propose the Class Desensitization Loss by employing contrastive learning to correct edge biases caused by data imbalance. Moreover, we dynamically fuse the weight parameters of the old and new models to achieve a better trade-off between the biased and unbiased model weights. To investigate the data imbalance problem in surgical scenarios, we construct a new benchmark for surgical instrument CSS by integrating four public datasets: EndoVis 2017, EndoVis 2018, CholecSeg8k, and SAR-RAPR50. Extensive experiments demonstrate the effectiveness of the proposed framework, achieving significant performance improvement against existing baselines. Our method demonstrates excellent potential for clinical applications. The code is publicly available at github.com/Zzsf11/SurgCSS.

Abstract Image

对类增量式手术器械分割数据不平衡的再思考
在手术器械分割中,随着时间的推移,越来越多的器械种类给现有的神经网络带来了重大挑战,因为它们无法有效地学习这些增量任务并遭受灾难性遗忘。当学习新数据时,模型在先前学习的数据上的性能会急剧下降。尽管已经提出了几种持续学习方法用于外科场景中的增量理解任务,但数据不平衡的问题通常会导致分割头部的强烈偏差,从而导致性能不佳。数据不平衡可以出现两种形式:(i)新旧数据之间的类不平衡,(ii)数据同一时间点内的类不平衡。这种不平衡通常会导致主导类接管持续语义分割(CSS)的训练过程。为了解决这个问题,我们提出了一个新的即插即用的CSS框架,用于数据不平衡下的手术器械分割。具体来说,我们通过图像绘制生成真实的手术背景,并以类感知的方式将仪器前景与生成的背景混合,以平衡各种场景下的数据分布。我们进一步提出了类脱敏损失,利用对比学习来纠正由数据不平衡引起的边缘偏差。此外,我们还动态融合了新旧模型的权重参数,以实现有偏和无偏模型权重之间更好的权衡。为了研究手术场景下的数据不平衡问题,我们整合了EndoVis 2017、EndoVis 2018、CholecSeg8k和SAR-RAPR50四个公共数据集,构建了一个新的手术器械CSS基准。大量的实验证明了所提出的框架的有效性,相对于现有基线实现了显着的性能改进。我们的方法在临床应用中具有很好的潜力。该代码可在github.com/Zzsf11/SurgCSS上公开获得。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Medical image analysis
Medical image analysis 工程技术-工程:生物医学
CiteScore
22.10
自引率
6.40%
发文量
309
审稿时长
6.6 months
期刊介绍: Medical Image Analysis serves as a platform for sharing new research findings in the realm of medical and biological image analysis, with a focus on applications of computer vision, virtual reality, and robotics to biomedical imaging challenges. The journal prioritizes the publication of high-quality, original papers contributing to the fundamental science of processing, analyzing, and utilizing medical and biological images. It welcomes approaches utilizing biomedical image datasets across all spatial scales, from molecular/cellular imaging to tissue/organ imaging.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信