FetalFlex: Anatomy-guided diffusion model for flexible control on fetal ultrasound image synthesis

IF 11.8 1区 医学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE
Yaofei Duan , Tao Tan , Zhiyuan Zhu , Yuhao Huang , Yuanji Zhang , Rui Gao , Patrick Cheong-Iao Pang , Xinru Gao , Guowei Tao , Xiang Cong , Zhou Li , Lianying Liang , Guangzhi He , Linliang Yin , Xuedong Deng , Xin Yang , Dong Ni
{"title":"FetalFlex: Anatomy-guided diffusion model for flexible control on fetal ultrasound image synthesis","authors":"Yaofei Duan ,&nbsp;Tao Tan ,&nbsp;Zhiyuan Zhu ,&nbsp;Yuhao Huang ,&nbsp;Yuanji Zhang ,&nbsp;Rui Gao ,&nbsp;Patrick Cheong-Iao Pang ,&nbsp;Xinru Gao ,&nbsp;Guowei Tao ,&nbsp;Xiang Cong ,&nbsp;Zhou Li ,&nbsp;Lianying Liang ,&nbsp;Guangzhi He ,&nbsp;Linliang Yin ,&nbsp;Xuedong Deng ,&nbsp;Xin Yang ,&nbsp;Dong Ni","doi":"10.1016/j.media.2025.103725","DOIUrl":null,"url":null,"abstract":"<div><div>Fetal ultrasound (US) examinations require the acquisition of multiple planes, each providing unique diagnostic information to evaluate fetal development and screening for congenital anomalies. However, obtaining a thorough, multi-plane annotated fetal US dataset remains challenging, particularly for rare or complex anomalies owing to their low incidence and numerous subtypes. This poses difficulties in training novice radiologists and developing robust AI models, especially for detecting abnormal fetuses. In this study, we introduce a Flexible Fetal US image generation framework (FetalFlex) to address these challenges, which leverages anatomical structures and multimodal information to enable controllable synthesis of fetal US images across diverse planes. Specifically, FetalFlex incorporates a pre-alignment module to enhance controllability and introduces a repaint strategy to ensure consistent texture and appearance. Moreover, a two-stage adaptive sampling strategy is developed to progressively refine image quality from coarse to fine levels. We believe that FetalFlex is the first method capable of generating both in-distribution normal and out-of-distribution abnormal fetal US images, without requiring any abnormal data. Experiments on multi-center datasets demonstrate that FetalFlex achieved state-of-the-art performance across multiple image quality metrics. Comprehensive reader studies further confirms the close alignment of the generated results with expert visual assessments and clinical-level fidelity. Furthermore, synthetic images by FetalFlex significantly improve the performance of six typical deep models in downstream classification and anomaly detection tasks. Lastly, FetalFlex’s anatomy-level controllable generation offers a unique advantage for anomaly simulation and creating paired or counterfactual data at the pixel level. The demo is available at: <span><span>https://dyf1023.github.io/FetalFlex/</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":18328,"journal":{"name":"Medical image analysis","volume":"105 ","pages":"Article 103725"},"PeriodicalIF":11.8000,"publicationDate":"2025-07-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Medical image analysis","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1361841525002725","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0

Abstract

Fetal ultrasound (US) examinations require the acquisition of multiple planes, each providing unique diagnostic information to evaluate fetal development and screening for congenital anomalies. However, obtaining a thorough, multi-plane annotated fetal US dataset remains challenging, particularly for rare or complex anomalies owing to their low incidence and numerous subtypes. This poses difficulties in training novice radiologists and developing robust AI models, especially for detecting abnormal fetuses. In this study, we introduce a Flexible Fetal US image generation framework (FetalFlex) to address these challenges, which leverages anatomical structures and multimodal information to enable controllable synthesis of fetal US images across diverse planes. Specifically, FetalFlex incorporates a pre-alignment module to enhance controllability and introduces a repaint strategy to ensure consistent texture and appearance. Moreover, a two-stage adaptive sampling strategy is developed to progressively refine image quality from coarse to fine levels. We believe that FetalFlex is the first method capable of generating both in-distribution normal and out-of-distribution abnormal fetal US images, without requiring any abnormal data. Experiments on multi-center datasets demonstrate that FetalFlex achieved state-of-the-art performance across multiple image quality metrics. Comprehensive reader studies further confirms the close alignment of the generated results with expert visual assessments and clinical-level fidelity. Furthermore, synthetic images by FetalFlex significantly improve the performance of six typical deep models in downstream classification and anomaly detection tasks. Lastly, FetalFlex’s anatomy-level controllable generation offers a unique advantage for anomaly simulation and creating paired or counterfactual data at the pixel level. The demo is available at: https://dyf1023.github.io/FetalFlex/.
FetalFlex:解剖导向的扩散模型,用于胎儿超声图像合成的柔性控制
胎儿超声(US)检查需要获取多个平面,每个平面提供独特的诊断信息,以评估胎儿发育和筛查先天性异常。然而,获得一个完整的、多平面注释的胎儿US数据集仍然具有挑战性,特别是对于罕见或复杂的异常,因为它们的发病率低且亚型众多。这给培训新手放射科医生和开发强大的人工智能模型带来了困难,特别是在检测异常胎儿方面。在这项研究中,我们引入了一个灵活的胎儿超声图像生成框架(FetalFlex)来解决这些挑战,它利用解剖结构和多模态信息来实现跨不同平面的胎儿超声图像的可控合成。具体来说,FetalFlex集成了一个预校准模块,以增强可控性,并引入了重涂策略,以确保纹理和外观的一致性。此外,开发了一种两阶段自适应采样策略,逐步将图像质量从粗级细化到精细级。我们认为FetalFlex是第一个能够同时生成分布内正常和分布外异常胎儿US图像的方法,而不需要任何异常数据。在多中心数据集上的实验表明,FetalFlex在多个图像质量指标上实现了最先进的性能。全面的读者研究进一步证实了与专家视觉评估和临床级保真度产生的结果的密切对齐。此外,FetalFlex合成图像显著提高了6种典型深度模型在下游分类和异常检测任务中的性能。最后,FetalFlex的解剖级可控生成为异常模拟和在像素级创建成对或反事实数据提供了独特的优势。该演示可在https://dyf1023.github.io/FetalFlex/上获得。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Medical image analysis
Medical image analysis 工程技术-工程:生物医学
CiteScore
22.10
自引率
6.40%
发文量
309
审稿时长
6.6 months
期刊介绍: Medical Image Analysis serves as a platform for sharing new research findings in the realm of medical and biological image analysis, with a focus on applications of computer vision, virtual reality, and robotics to biomedical imaging challenges. The journal prioritizes the publication of high-quality, original papers contributing to the fundamental science of processing, analyzing, and utilizing medical and biological images. It welcomes approaches utilizing biomedical image datasets across all spatial scales, from molecular/cellular imaging to tissue/organ imaging.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信