IEEE transactions on medical imaging最新文献_第3页

Training-Free Image Style Alignment for Domain Shift on Handheld Ultrasound Devices 在手持式超声设备上进行域移位的无训练图像样式对齐

IEEE transactions on medical imaging Pub Date : 2024-12-25 DOI: 10.1109/TMI.2024.3522071

Hongye Zeng;Ke Zou;Zhihao Chen;Yuchong Gao;Hongbo Chen;Haibin Zhang;Kang Zhou;Meng Wang;Chang Jiang;Rick Siow Mong Goh;Yong Liu;Chengcheng Zhu;Rui Zheng;Huazhu Fu

{"title":"Training-Free Image Style Alignment for Domain Shift on Handheld Ultrasound Devices","authors":"Hongye Zeng;Ke Zou;Zhihao Chen;Yuchong Gao;Hongbo Chen;Haibin Zhang;Kang Zhou;Meng Wang;Chang Jiang;Rick Siow Mong Goh;Yong Liu;Chengcheng Zhu;Rui Zheng;Huazhu Fu","doi":"10.1109/TMI.2024.3522071","DOIUrl":"10.1109/TMI.2024.3522071","url":null,"abstract":"Handheld ultrasound devices face usage limitations due to user inexperience and cannot benefit from supervised deep learning without extensive expert annotations. Moreover, the models trained on standard ultrasound device data are constrained by training data distribution and perform poorly when directly applied to handheld device data. In this study, we propose the Training-free Image Style Alignment (TISA) to align the style of handheld device data to those of standard devices. The proposed TISA eliminates the demand for source data, and can transform the image style while preserving spatial context during testing. Furthermore, our TISA avoids continuous updates to the pre-trained model compared to other test-time methods and is suited for clinical applications. We show that TISA performs better and more stably in medical detection and segmentation tasks for handheld device data than other test-time adaptation methods. We further validate TISA as the clinical model for automatic measurements of spinal curvature and carotid intima-media thickness, and the automatic measurements agree well with manual measurements made by human experts. We demonstrate the potential for TISA to facilitate automatic diagnosis on handheld ultrasound devices and expedite their eventual widespread use. Code is available at <uri>https://github.com/zenghy96/TISA</uri>.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 4","pages":"1942-1952"},"PeriodicalIF":0.0,"publicationDate":"2024-12-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142887406","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Segmenting the Inferior Alveolar Canal in CBCTs Volumes: The ToothFairy Challenge 在cbct中分割下牙槽管：牙齿仙女的挑战

IEEE transactions on medical imaging Pub Date : 2024-12-25 DOI: 10.1109/TMI.2024.3523096

Federico Bolelli;Luca Lumetti;Shankeeth Vinayahalingam;Mattia Di Bartolomeo;Arrigo Pellacani;Kevin Marchesini;Niels van Nistelrooij;Pieter van Lierop;Tong Xi;Yusheng Liu;Rui Xin;Tao Yang;Lisheng Wang;Haoshen Wang;Chenfan Xu;Zhiming Cui;Marek Wodzinski;Henning Müller;Yannick Kirchhoff;Maximilian R. Rokuss;Klaus Maier-Hein;Jaehwan Han;Wan Kim;Hong-Gi Ahn;Tomasz Szczepański;Michal K. Grzeszczyk;Przemyslaw Korzeniowski;Vicent Caselles-Ballester;Xavier Paolo Burgos-Artizzu;Ferran Prados Carrasco;Stefaan Berge’;Bram van Ginneken;Alexandre Anesi;Costantino Grana

{"title":"Segmenting the Inferior Alveolar Canal in CBCTs Volumes: The ToothFairy Challenge","authors":"Federico Bolelli;Luca Lumetti;Shankeeth Vinayahalingam;Mattia Di Bartolomeo;Arrigo Pellacani;Kevin Marchesini;Niels van Nistelrooij;Pieter van Lierop;Tong Xi;Yusheng Liu;Rui Xin;Tao Yang;Lisheng Wang;Haoshen Wang;Chenfan Xu;Zhiming Cui;Marek Wodzinski;Henning Müller;Yannick Kirchhoff;Maximilian R. Rokuss;Klaus Maier-Hein;Jaehwan Han;Wan Kim;Hong-Gi Ahn;Tomasz Szczepański;Michal K. Grzeszczyk;Przemyslaw Korzeniowski;Vicent Caselles-Ballester;Xavier Paolo Burgos-Artizzu;Ferran Prados Carrasco;Stefaan Berge’;Bram van Ginneken;Alexandre Anesi;Costantino Grana","doi":"10.1109/TMI.2024.3523096","DOIUrl":"10.1109/TMI.2024.3523096","url":null,"abstract":"In recent years, several algorithms have been developed for the segmentation of the Inferior Alveolar Canal (IAC) in Cone-Beam Computed Tomography (CBCT) scans. However, the availability of public datasets in this domain is limited, resulting in a lack of comparative evaluation studies on a common benchmark. To address this scientific gap and encourage deep learning research in the field, the ToothFairy challenge was organized within the MICCAI 2023 conference. In this context, a public dataset was released to also serve as a benchmark for future research. The dataset comprises 443 CBCT scans, with voxel-level annotations of the IAC available for 153 of them, making it the largest publicly available dataset of its kind. The participants of the challenge were tasked with developing an algorithm to accurately identify the IAC using the 2D and 3D-annotated scans. This paper presents the details of the challenge and the contributions made by the most promising methods proposed by the participants. It represents the first comprehensive comparative evaluation of IAC segmentation methods on a common benchmark dataset, providing insights into the current state-of-the-art algorithms and outlining future research directions. Furthermore, to ensure reproducibility and promote future developments, an open-source repository that collects the implementations of the best submissions was released.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 4","pages":"1890-1906"},"PeriodicalIF":0.0,"publicationDate":"2024-12-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10816445","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142887642","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Reliable 2D–3D Registration in Dynamic Stereo-Radiography With Energy Barrier Constraints 具有能量势垒约束的动态立体放射成像中可靠的2D-3D配准

IEEE transactions on medical imaging Pub Date : 2024-12-24 DOI: 10.1109/TMI.2024.3522200

William S. Burton;Casey A. Myers;Chadd C. Clary;Paul J. Rullkoetter

{"title":"Reliable 2D–3D Registration in Dynamic Stereo-Radiography With Energy Barrier Constraints","authors":"William S. Burton;Casey A. Myers;Chadd C. Clary;Paul J. Rullkoetter","doi":"10.1109/TMI.2024.3522200","DOIUrl":"10.1109/TMI.2024.3522200","url":null,"abstract":"2D-3D registration of native anatomy in dynamic stereo-radiography is a fundamental task in orthopaedics methods that facilitates understanding of joint-level movement. Registration is commonly performed by optimizing a similarity metric which compares the appearances of captured radiographs to computed tomography-based digitally reconstructed radiographs, rendered as a function of pose. This optimization-based framework can accurately recover the pose of native anatomy in stereo-radiographs, but encounters convergence issues in practice, thus limiting the reliability of fully automatic registration. The current work improves the robustness of optimization-based 2D-3D registration through the introduction of data-driven constraints that restrict the set of evaluated pose candidates. Energy-based models are first developed to indicate the viability of anatomic poses, conditioned on target radiographs. Registration is then performed by ensuring that optimization methods search within regions that contain feasible poses, as dictated by energy-based models. The constraints which define these regions of interest are referred to as Energy Barrier Constraints. Experiments with stereo-radiographs capturing glenohumeral anatomy were performed to evaluate the proposed methods. Mean errors of 3.2-5.3 and 2.4-4.8 degrees or mm were observed for scapula and humerus degrees of freedom, respectively, when optimizing a conventional similarity metric. These errors were improved to 0.2-0.7 and 0.4-4.1 degrees or mm when augmenting the similarity metric with the proposed techniques. Results suggest that the introduced methods may benefit optimization-based 2D-3D registration through improved reliability.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 4","pages":"1969-1983"},"PeriodicalIF":0.0,"publicationDate":"2024-12-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142884383","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

PASS: Test-Time Prompting to Adapt Styles and Semantic Shapes in Medical Image Segmentation 医学图像分割中适应样式和语义形状的测试时间提示

IEEE transactions on medical imaging Pub Date : 2024-12-23 DOI: 10.1109/TMI.2024.3521463

Chuyan Zhang;Hao Zheng;Xin You;Yefeng Zheng;Yun Gu

{"title":"PASS: Test-Time Prompting to Adapt Styles and Semantic Shapes in Medical Image Segmentation","authors":"Chuyan Zhang;Hao Zheng;Xin You;Yefeng Zheng;Yun Gu","doi":"10.1109/TMI.2024.3521463","DOIUrl":"10.1109/TMI.2024.3521463","url":null,"abstract":"Test-time adaptation (TTA) has emerged as a promising paradigm to handle the domain shifts at test time for medical images from different institutions without using extra training data. However, existing TTA solutions for segmentation tasks suffer from 1) dependency on modifying the source training stage and access to source priors or 2) lack of emphasis on shape-related semantic knowledge that is crucial for segmentation tasks. Recent research on visual prompt learning achieves source-relaxed adaptation by extended parameter space but still neglects the full utilization of semantic features, thus motivating our work on knowledge-enriched deep prompt learning. Beyond the general concern of image style shifts, we reveal that shape variability is another crucial factor causing the performance drop. To address this issue, we propose a TTA framework called PASS (Prompting to Adapt Styles and Semantic shapes), which jointly learns two types of prompts: the input-space prompt to reformulate the style of the test image to fit into the pretrained model and the semantic-aware prompts to bridge high-level shape discrepancy across domains. Instead of naively imposing a fixed prompt, we introduce an input decorator to generate the self-regulating visual prompt conditioned on the input data. To retrieve the knowledge representations and customize target-specific shape prompts for each test sample, we propose a cross-attention prompt modulator, which performs interaction between target representations and an enriched shape prompt bank. Extensive experiments demonstrate the superior performance of PASS over state-of-the-art methods on multiple medical image segmentation datasets. The code is available at <uri>https://github.com/EndoluminalSurgicalVision-IMR/PASS</uri>.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 4","pages":"1853-1865"},"PeriodicalIF":0.0,"publicationDate":"2024-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142879592","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MultiEYE: Dataset and Benchmark for OCT-Enhanced Retinal Disease Recognition From Fundus Images MultiEYE：眼底图像oct增强视网膜疾病识别的数据集和基准

IEEE transactions on medical imaging Pub Date : 2024-12-23 DOI: 10.1109/TMI.2024.3518067

Lehan Wang;Chongchong Qi;Chubin Ou;Lin An;Mei Jin;Xiangbin Kong;Xiaomeng Li

{"title":"MultiEYE: Dataset and Benchmark for OCT-Enhanced Retinal Disease Recognition From Fundus Images","authors":"Lehan Wang;Chongchong Qi;Chubin Ou;Lin An;Mei Jin;Xiangbin Kong;Xiaomeng Li","doi":"10.1109/TMI.2024.3518067","DOIUrl":"10.1109/TMI.2024.3518067","url":null,"abstract":"Existing multi-modal learning methods on fundus and OCT images mostly require both modalities to be available and strictly paired for training and testing, which appears less practical in clinical scenarios. To expand the scope of clinical applications, we formulate a novel setting, “OCT-enhanced disease recognition from fundus images”, that allows for the use of unpaired multi-modal data during the training phase, and relies on the widespread fundus photographs for testing. To benchmark this setting, we present the first large multi-modal multi-class dataset for eye disease diagnosis, MultiEYE, and propose an OCT-assisted Conceptual Distillation Approach (OCT-CoDA), which employs semantically rich concepts to extract disease-related knowledge from OCT images and leverages them into the fundus model. Specifically, we regard the image-concept relation as a link to distill useful knowledge from OCT teacher model to fundus student model, which considerably improves the diagnostic performance based on fundus images and formulates the cross-modal knowledge transfer into an explainable process. Through extensive experiments on the multi-disease classification task, our proposed OCT-CoDA demonstrates remarkable results and interpretability, showing great potential for clinical application. Our dataset and code are available at <uri>https://github.com/xmed-lab/MultiEYE</uri>.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 4","pages":"1711-1722"},"PeriodicalIF":0.0,"publicationDate":"2024-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142879942","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Task-Oriented Network Design for Visual Tracking and Motion Filtering of Needle Tip Under 2D Ultrasound 面向任务的二维超声针尖视觉跟踪与运动滤波网络设计

IEEE transactions on medical imaging Pub Date : 2024-12-23 DOI: 10.1109/TMI.2024.3520992

Wanquan Yan;Raymond Shing-Yan Tang;Shing Shin Cheng

{"title":"Task-Oriented Network Design for Visual Tracking and Motion Filtering of Needle Tip Under 2D Ultrasound","authors":"Wanquan Yan;Raymond Shing-Yan Tang;Shing Shin Cheng","doi":"10.1109/TMI.2024.3520992","DOIUrl":"10.1109/TMI.2024.3520992","url":null,"abstract":"Needle tip tracking under ultrasound (US) imaging is critical for accurate lesion targeting in US-guided percutaneous procedures. While most state-of-the-art trackers have relied on complex network architecture for enhanced performance, the compromised computational efficiency prevents their real-time implementation. Pure visual trackers are also limited in addressing the drift errors caused by temporary needle tip disappearance. In this paper, a compact, task-oriented visual tracker, consisting of an appearance adaptation module and a distractor suppression module, is first designed before it is integrated with a motion filter, namely TransKalman, that leverages the Transformer network for Kalman filter gain estimation. The ablation study shows that the mean tracking success rate (i.e. error <3mm in 95% video frames) of the visual tracker increases by 25% compared with its baseline model. The complete tracking system, integrating the visual tracker and TransKalman, outperforms other existing trackers by at least 5.1% in success rate and 47% in tracking speed during manual needle manipulation experiments in ex-vivo tissue. The proposed real-time tracking system will potentially be integrated in both manual and robotic procedures to reduce operator dependence and improve targeting accuracy during needle-based diagnostic and therapeutic procedures.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 4","pages":"1735-1749"},"PeriodicalIF":0.0,"publicationDate":"2024-12-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10811967","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142879874","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

HMIL: Hierarchical Multi-Instance Learning for Fine-Grained Whole Slide Image Classification 层次多实例学习用于细粒度整张幻灯片图像分类

IEEE transactions on medical imaging Pub Date : 2024-12-20 DOI: 10.1109/TMI.2024.3520602

Cheng Jin;Luyang Luo;Huangjing Lin;Jun Hou;Hao Chen

{"title":"HMIL: Hierarchical Multi-Instance Learning for Fine-Grained Whole Slide Image Classification","authors":"Cheng Jin;Luyang Luo;Huangjing Lin;Jun Hou;Hao Chen","doi":"10.1109/TMI.2024.3520602","DOIUrl":"10.1109/TMI.2024.3520602","url":null,"abstract":"Fine-grained classification of whole slide images (WSIs) is essential in precision oncology, enabling precise cancer diagnosis and personalized treatment strategies. The core of this task involves distinguishing subtle morphological variations within the same broad category of gigapixel-resolution images, which presents a significant challenge. While the multi-instance learning (MIL) paradigm alleviates the computational burden of WSIs, existing MIL methods often overlook hierarchical label correlations, treating fine-grained classification as a flat multi-class classification task. To overcome these limitations, we introduce a novel hierarchical multi-instance learning (HMIL) framework. By facilitating on the hierarchical alignment of inherent relationships between different hierarchy of labels at instance and bag level, our approach provides a more structured and informative learning process. Specifically, HMIL incorporates a class-wise attention mechanism that aligns hierarchical information at both the instance and bag levels. Furthermore, we introduce supervised contrastive learning to enhance the discriminative capability for fine-grained classification and a curriculum-based dynamic weighting module to adaptively balance the hierarchical feature during training. Extensive experiments on our large-scale cytology cervical cancer (CCC) dataset and two public histology datasets, BRACS and PANDA, demonstrate the state-of-the-art class-wise and overall performance of our HMIL framework. Our source code is available at <uri>https://github.com/ChengJin-git/HMIL</uri>.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 4","pages":"1796-1808"},"PeriodicalIF":0.0,"publicationDate":"2024-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142866968","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

DEeR: Deviation Eliminating and Noise Regulating for Privacy-Preserving Federated Low-Rank Adaptation 基于隐私保护的联邦低秩自适应的偏差消除和噪声调节

IEEE transactions on medical imaging Pub Date : 2024-12-19 DOI: 10.1109/TMI.2024.3518539

Meilu Zhu;Axiu Mao;Jun Liu;Yixuan Yuan

{"title":"DEeR: Deviation Eliminating and Noise Regulating for Privacy-Preserving Federated Low-Rank Adaptation","authors":"Meilu Zhu;Axiu Mao;Jun Liu;Yixuan Yuan","doi":"10.1109/TMI.2024.3518539","DOIUrl":"10.1109/TMI.2024.3518539","url":null,"abstract":"Integrating low-rank adaptation (LoRA) with federated learning (FL) has received widespread attention recently, aiming to adapt pretrained foundation models (FMs) to downstream medical tasks via privacy-preserving decentralized training. However, owing to the direct combination of LoRA and FL, current methods generally undergo two problems, i.e., aggregation deviation, and differential privacy (DP) noise amplification effect. To address these problems, we propose a novel privacy-preserving federated finetuning framework called Deviation Eliminating and Noise Regulating (DEeR). Specifically, we firstly theoretically prove that the necessary condition to eliminate aggregation deviation is guaranteeing the equivalence between LoRA parameters of clients. Based on the theoretical insight, a deviation eliminator is designed to utilize alternating minimization algorithm to iteratively optimize the zero-initialized and non-zero-initialized parameter matrices of LoRA, ensuring that aggregation deviation always be zeros during training. Furthermore, we also conduct an in-depth analysis of the noise amplification effect and find that this problem is mainly caused by the “linear relationship” between DP noise and LoRA parameters. To suppress the noise amplification effect, we propose a noise regulator that exploits two regulator factors to decouple relationship between DP and LoRA, thereby achieving robust privacy protection and excellent finetuning performance. Additionally, we perform comprehensive ablated experiments to verify the effectiveness of the deviation eliminator and noise regulator. DEeR shows better performance on public medical datasets in comparison with state-of-the-art approaches. The code is available at <uri>https://github.com/CUHK-AIM-Group/DEeR</uri>.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 4","pages":"1783-1795"},"PeriodicalIF":0.0,"publicationDate":"2024-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142858423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

HisynSeg: Weakly-Supervised Histopathological Image Segmentation via Image-Mixing Synthesis and Consistency Regularization HisynSeg：基于图像混合合成和一致性正则化的弱监督组织病理图像分割

IEEE transactions on medical imaging Pub Date : 2024-12-19 DOI: 10.1109/TMI.2024.3520129

Zijie Fang;Yifeng Wang;Peizhang Xie;Zhi Wang;Yongbing Zhang

{"title":"HisynSeg: Weakly-Supervised Histopathological Image Segmentation via Image-Mixing Synthesis and Consistency Regularization","authors":"Zijie Fang;Yifeng Wang;Peizhang Xie;Zhi Wang;Yongbing Zhang","doi":"10.1109/TMI.2024.3520129","DOIUrl":"10.1109/TMI.2024.3520129","url":null,"abstract":"Tissue semantic segmentation is one of the key tasks in computational pathology. To avoid the expensive and laborious acquisition of pixel-level annotations, a wide range of studies attempt to adopt the class activation map (CAM), a weakly-supervised learning scheme, to achieve pixel-level tissue segmentation. However, CAM-based methods are prone to suffer from under-activation and over-activation issues, leading to poor segmentation performance. To address this problem, we propose a novel weakly-supervised semantic segmentation framework for histopathological images based on image-mixing synthesis and consistency regularization, dubbed HisynSeg. Specifically, synthesized histopathological images with pixel-level masks are generated for fully-supervised model training, where two synthesis strategies are proposed based on Mosaic transformation and Bézier mask generation. Besides, an image filtering module is developed to guarantee the authenticity of the synthesized images. In order to further avoid the model overfitting to the occasional synthesis artifacts, we additionally propose a novel self-supervised consistency regularization, which enables the real images without segmentation masks to supervise the training of the segmentation model. By integrating the proposed techniques, the HisynSeg framework successfully transforms the weakly-supervised semantic segmentation problem into a fully-supervised one, greatly improving the segmentation accuracy. Experimental results on three datasets prove that the proposed method achieves a state-of-the-art performance. Code is available at <uri>https://github.com/Vison307/HisynSeg</uri>.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 4","pages":"1765-1782"},"PeriodicalIF":0.0,"publicationDate":"2024-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142858424","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Unlocking the Potential of Weakly Labeled Data: A Co-Evolutionary Learning Framework for Abnormality Detection and Report Generation 释放弱标记数据的潜力：异常检测和报告生成的协同进化学习框架

IEEE transactions on medical imaging Pub Date : 2024-12-13 DOI: 10.1109/TMI.2024.3516954

Jinghan Sun;Dong Wei;Zhe Xu;Donghuan Lu;Hong Liu;Hong Wang;Sotirios A. Tsaftaris;Steven McDonagh;Yefeng Zheng;Liansheng Wang

{"title":"Unlocking the Potential of Weakly Labeled Data: A Co-Evolutionary Learning Framework for Abnormality Detection and Report Generation","authors":"Jinghan Sun;Dong Wei;Zhe Xu;Donghuan Lu;Hong Liu;Hong Wang;Sotirios A. Tsaftaris;Steven McDonagh;Yefeng Zheng;Liansheng Wang","doi":"10.1109/TMI.2024.3516954","DOIUrl":"10.1109/TMI.2024.3516954","url":null,"abstract":"Anatomical abnormality detection and report generation of chest X-ray (CXR) are two essential tasks in clinical practice. The former aims at localizing and characterizing cardiopulmonary radiological findings in CXRs, while the latter summarizes the findings in a detailed report for further diagnosis and treatment. Existing methods often focused on either task separately, ignoring their correlation. This work proposes a co-evolutionary abnormality detection and report generation (CoE-DG) framework. The framework utilizes both fully labeled (with bounding box annotations and clinical reports) and weakly labeled (with reports only) data to achieve mutual promotion between the abnormality detection and report generation tasks. Specifically, we introduce a bi-directional information interaction strategy with generator-guided information propagation (GIP) and detector-guided information propagation (DIP). For semi-supervised abnormality detection, GIP takes the informative feature extracted by the generator as an auxiliary input to the detector and uses the generator’s prediction to refine the detector’s pseudo labels. We further propose an intra-image-modal self-adaptive non-maximum suppression module (SA-NMS). This module dynamically rectifies pseudo detection labels generated by the teacher detection model with high-confidence predictions by the student. Inversely, for report generation, DIP takes the abnormalities’ categories and locations predicted by the detector as input and guidance for the generator to improve the generated reports. Finally, a co-evolutionary training strategy is implemented to iteratively conduct GIP and DIP and consistently improve both tasks’ performance. Experimental results on two public CXR datasets demonstrate CoE-DG’s superior performance to several up-to-date object detection, report generation, and unified models. Our code is available at <uri>https://github.com/jinghanSunn/CoE-DG</uri>.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 4","pages":"1671-1685"},"PeriodicalIF":0.0,"publicationDate":"2024-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142820730","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0