{"title":"三维MRI扩散模型合成超声跨模态半监督迁移学习分割脑肿瘤","authors":"Yuhua Li , Shan Jiang, Zhiyong Yang, Liwen Wang, Shuangying Wang, Zeyang Zhou","doi":"10.1016/j.inffus.2025.103757","DOIUrl":null,"url":null,"abstract":"<div><div>Accurate ultrasound segmentation is crucial for intraoperative brain navigation and can improve non-rigid registration between preoperative MRI and intraoperative ultrasound, compensating for brain shift. However, limited annotated ultrasound data hinder the application of deep learning methods. Given recent advances in brain MRI-based medical image processing, transferring MRI datasets and deep learning models to US image research via cross-modal translation may potentially enhance intelligent brain US image processing. In this paper, we propose a novel cross-modality semi-supervised transfer learning from MRI to US by leveraging annotated data in the MRI modality. A diffusion model, leveraging conditional texture features and guided mutual information, transforms well-annotated MRI images into synthetic US images with a distribution closer to real US images. Subsequently, we employ a segmentation framework that involves pretraining with synthetic US images derived from MRI through image translation, followed by semi-supervised fine-tuning using a hybrid dataset that integrates both labeled and unlabeled ultrasound data. Extensive assessments are reported on the utility of SL-DDPM against competing GAN and diffusion models in MRI-US translation. The experimental results demonstrate that our proposed transfer learning strategy achieves a segmentation accuracy of DSC of 93.43 ± 3.72 %. The effectiveness of our strategy is validated through ablation studies on fine-tuning strategies and semi-supervised learning, as well as comparisons with other state-of-the-art methods. Our transfer learning strategy enhances the accuracy and generalization of brain ultrasound segmentation models, even with limited hybrid training data, thereby assisting surgeons in identifying lesion.</div></div>","PeriodicalId":50367,"journal":{"name":"Information Fusion","volume":"127 ","pages":"Article 103757"},"PeriodicalIF":15.5000,"publicationDate":"2025-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Brain tumor segmentation via cross-modality semi-supervised transfer learning with 3D MRI diffusion model synthetic ultrasound\",\"authors\":\"Yuhua Li , Shan Jiang, Zhiyong Yang, Liwen Wang, Shuangying Wang, Zeyang Zhou\",\"doi\":\"10.1016/j.inffus.2025.103757\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Accurate ultrasound segmentation is crucial for intraoperative brain navigation and can improve non-rigid registration between preoperative MRI and intraoperative ultrasound, compensating for brain shift. However, limited annotated ultrasound data hinder the application of deep learning methods. Given recent advances in brain MRI-based medical image processing, transferring MRI datasets and deep learning models to US image research via cross-modal translation may potentially enhance intelligent brain US image processing. In this paper, we propose a novel cross-modality semi-supervised transfer learning from MRI to US by leveraging annotated data in the MRI modality. A diffusion model, leveraging conditional texture features and guided mutual information, transforms well-annotated MRI images into synthetic US images with a distribution closer to real US images. Subsequently, we employ a segmentation framework that involves pretraining with synthetic US images derived from MRI through image translation, followed by semi-supervised fine-tuning using a hybrid dataset that integrates both labeled and unlabeled ultrasound data. Extensive assessments are reported on the utility of SL-DDPM against competing GAN and diffusion models in MRI-US translation. The experimental results demonstrate that our proposed transfer learning strategy achieves a segmentation accuracy of DSC of 93.43 ± 3.72 %. The effectiveness of our strategy is validated through ablation studies on fine-tuning strategies and semi-supervised learning, as well as comparisons with other state-of-the-art methods. Our transfer learning strategy enhances the accuracy and generalization of brain ultrasound segmentation models, even with limited hybrid training data, thereby assisting surgeons in identifying lesion.</div></div>\",\"PeriodicalId\":50367,\"journal\":{\"name\":\"Information Fusion\",\"volume\":\"127 \",\"pages\":\"Article 103757\"},\"PeriodicalIF\":15.5000,\"publicationDate\":\"2025-09-16\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Information Fusion\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S156625352500819X\",\"RegionNum\":1,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Information Fusion","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S156625352500819X","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
Brain tumor segmentation via cross-modality semi-supervised transfer learning with 3D MRI diffusion model synthetic ultrasound
Accurate ultrasound segmentation is crucial for intraoperative brain navigation and can improve non-rigid registration between preoperative MRI and intraoperative ultrasound, compensating for brain shift. However, limited annotated ultrasound data hinder the application of deep learning methods. Given recent advances in brain MRI-based medical image processing, transferring MRI datasets and deep learning models to US image research via cross-modal translation may potentially enhance intelligent brain US image processing. In this paper, we propose a novel cross-modality semi-supervised transfer learning from MRI to US by leveraging annotated data in the MRI modality. A diffusion model, leveraging conditional texture features and guided mutual information, transforms well-annotated MRI images into synthetic US images with a distribution closer to real US images. Subsequently, we employ a segmentation framework that involves pretraining with synthetic US images derived from MRI through image translation, followed by semi-supervised fine-tuning using a hybrid dataset that integrates both labeled and unlabeled ultrasound data. Extensive assessments are reported on the utility of SL-DDPM against competing GAN and diffusion models in MRI-US translation. The experimental results demonstrate that our proposed transfer learning strategy achieves a segmentation accuracy of DSC of 93.43 ± 3.72 %. The effectiveness of our strategy is validated through ablation studies on fine-tuning strategies and semi-supervised learning, as well as comparisons with other state-of-the-art methods. Our transfer learning strategy enhances the accuracy and generalization of brain ultrasound segmentation models, even with limited hybrid training data, thereby assisting surgeons in identifying lesion.
期刊介绍:
Information Fusion serves as a central platform for showcasing advancements in multi-sensor, multi-source, multi-process information fusion, fostering collaboration among diverse disciplines driving its progress. It is the leading outlet for sharing research and development in this field, focusing on architectures, algorithms, and applications. Papers dealing with fundamental theoretical analyses as well as those demonstrating their application to real-world problems will be welcome.