Heng Li;Mingyang Ou;Haojin Li;Zhongxi Qiu;Ke Niu;Huazhu Fu;Jiang Liu
{"title":"Multi-View Test-Time Adaptation for Semantic Segmentation in Clinical Cataract Surgery","authors":"Heng Li;Mingyang Ou;Haojin Li;Zhongxi Qiu;Ke Niu;Huazhu Fu;Jiang Liu","doi":"10.1109/TMI.2025.3529875","DOIUrl":"10.1109/TMI.2025.3529875","url":null,"abstract":"Cataract surgery, a widely performed operation worldwide, is incorporating semantic segmentation to advance computer-assisted intervention. However, the tissue appearance and illumination in cataract surgery often differ among clinical centers, intensifying the issue of domain shifts. While domain adaptation offers remedies to the shifts, the necessity for data centralization raises additional privacy concerns. To overcome these challenges, we propose a Multi-view Test-time Adaptation algorithm (MUTA) to segment cataract surgical scenes, which leverages multi-view learning to enhance model training within the source domain and model adaptation within the target domain. In the training phase, the segmentation model is equipped with multi-view decoders to boost its robustness against variations in cataract surgery. During the inference phase, test-time adaptation is implemented using multi-view knowledge distillation, enabling model updates in clinics without data centralization or privacy concerns. We conducted experiments in a simulated cross-center scenario using several cataract surgery datasets to evaluate the effectiveness of MUTA. Through comparisons and investigations, we have validated that MUTA effectively learns a robust source model and adapts the model to target data during the practical inference phase. Code and datasets are available at <uri>https://github.com/liamheng/CAI-algorithms</uri>.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 5","pages":"2307-2318"},"PeriodicalIF":0.0,"publicationDate":"2025-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142987448","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Toby Sanders;Justin Konkle;Olivia C. Sehl;A. Rahman Mohtasebzadeh;Joan M. Greve;Patrick W. Goodwill
{"title":"A Physics-Based Computational Forward Model for Efficient Image Reconstruction in Magnetic Particle Imaging","authors":"Toby Sanders;Justin Konkle;Olivia C. Sehl;A. Rahman Mohtasebzadeh;Joan M. Greve;Patrick W. Goodwill","doi":"10.1109/TMI.2025.3530316","DOIUrl":"10.1109/TMI.2025.3530316","url":null,"abstract":"This article derives and implements a computational physics model for model-based image reconstruction in magnetic particle imaging (MPI) applications. To our knowledge, this is the first ever computationally tractable model-based image reconstruction in MPI, which is neither constructed from calibration or simulation experiments or limited to specific scan acquisition geometries. The derived model results in a system constructed from a series of fast linear transforms, each of which incorporate the individual components from the paramagnetic model. These include the field free point velocity and location, gradient strength, receive coil sensitivity, and receive chain filtering. Each of these modeling components are amendable to any changes in the acquisition parameters. This allows us to adopt a computationally tractable system matrix modeling approach to MPI for any scan specific parameters at very high pixel resolutions. The model is derived from first principles, and it results from taking the fundamental MPI signal theory and decomposing these modeling equations into the series of linear transforms acting on a pixelated image. Each transform is formally defined in matrix form but implemented in a matrix-free fashion with fast and/or sparse operations. For these reasons, our new model should be a fundamental tool in the future of computational imaging in MPI. We demonstrate our new method on a variety of pre-clinical and simulated data sets, and these results confirm that our method is both efficient and accurate.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 5","pages":"2319-2329"},"PeriodicalIF":0.0,"publicationDate":"2025-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142986789","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yijun Yang;Huazhu Fu;Angelica I. Aviles-Rivero;Zhaohu Xing;Lei Zhu
{"title":"DiffMIC-v2: Medical Image Classification via Improved Diffusion Network","authors":"Yijun Yang;Huazhu Fu;Angelica I. Aviles-Rivero;Zhaohu Xing;Lei Zhu","doi":"10.1109/TMI.2025.3530399","DOIUrl":"10.1109/TMI.2025.3530399","url":null,"abstract":"Recently, Denoising Diffusion Models have achieved outstanding success in generative image modeling and attracted significant attention in the computer vision community. Although a substantial amount of diffusion-based research has focused on generative tasks, few studies apply diffusion models to medical diagnosis. In this paper, we propose a diffusion-based network (named DiffMIC-v2) to address general medical image classification by eliminating unexpected noise and perturbations in image representations. To achieve this goal, we first devise an improved dual-conditional guidance strategy that conditions each diffusion step with multiple granularities to enhance step-wise regional attention. Furthermore, we design a novel Heterologous diffusion process that achieves efficient visual representation learning in the latent space. We evaluate the effectiveness of our DiffMIC-v2 on four medical classification tasks with different image modalities, including thoracic diseases classification on chest X-ray, placental maturity grading on ultrasound images, skin lesion classification using dermatoscopic images, and diabetic retinopathy grading using fundus images. Experimental results demonstrate that our DiffMIC-v2 outperforms state-of-the-art methods by a significant margin, which indicates the universality and effectiveness of the proposed model on multi-class and multi-label classification tasks. DiffMIC-v2 can use fewer iterations than our previous DiffMIC to obtain accurate estimations, and also achieves greater runtime efficiency with superior results. The code will be publicly available at <uri>https://github.com/scott-yjyang/DiffMICv2</uri>.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 5","pages":"2244-2255"},"PeriodicalIF":0.0,"publicationDate":"2025-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142986790","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Chengpu Wei;Zhe Li;Ting Hu;Mengyang Zhao;Zhonghua Sun;Kebin Jia;Jinchao Feng;Brain W. Pogue;Keith D. Paulsen;Shudong Jiang
{"title":"Model-Based Convolution Neural Network for 3D Near-Infrared Spectral Tomography","authors":"Chengpu Wei;Zhe Li;Ting Hu;Mengyang Zhao;Zhonghua Sun;Kebin Jia;Jinchao Feng;Brain W. Pogue;Keith D. Paulsen;Shudong Jiang","doi":"10.1109/TMI.2025.3529621","DOIUrl":"10.1109/TMI.2025.3529621","url":null,"abstract":"Near-infrared spectral tomography (NIRST) is a non-invasive imaging technique that provides functional information about biological tissues. Due to diffuse light propagation in tissue and limited boundary measurements, NIRST image reconstruction presents an ill-posed and ill-conditioned computational problem that is difficult to solve. To address this challenge, we developed a reconstruction algorithm (Model-CNN) that integrates a diffusion equation model with a convolutional neural network (CNN). The CNN learns a regularization prior to restrict solutions to the space of desirable chromophore concentration images. Efficacy of Model-CNN was evaluated by training on numerical simulation data, and then applying the network to physical phantom and clinical patient NIRST data. Results demonstrated the superiority of Model-CNN over the conventional Tikhonov regularization approach and a deep learning algorithm (FC-CNN) in terms of absolute bias error (ABE) and peak signal-to-noise ratio (PSNR). Specifically, in comparison to Tikhonov regularization, Model-CNN reduced average ABE by 55% for total hemoglobin (HbT) and 70% water (H<inline-formula> <tex-math>$_{mathbf {{2}}}$ </tex-math></inline-formula> O) concentration, while improved PSNR by an average of 5.3 dB both for HbT and H<inline-formula> <tex-math>$_{mathbf {{2}}}$ </tex-math></inline-formula> O images. Meanwhile, image processing time was reduced by 82%, relative to the Tikhonov regularization. As compared to FC-CNN, the Model-CNN achieved a 91% reduction in ABE for HbT and 75% for H<inline-formula> <tex-math>$_{mathbf {{2}}}$ </tex-math></inline-formula> O images, with increases in PSNR by 7.3 dB and 4.7 dB, respectively. Notably, this Model-CNN approach was not trained on patient data; but instead, was trained on simulated phantom data with simpler geometrical shapes and optical source-detector configurations; yet, achieved superior image recovery when faced with real-world data.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 5","pages":"2330-2340"},"PeriodicalIF":0.0,"publicationDate":"2025-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142981394","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Knowledge-Driven Framework for Anatomical Landmark Annotation in Laparoscopic Surgery","authors":"Jie Zhang;Song Zhou;Yiwei Wang;Huan Zhao;Han Ding","doi":"10.1109/TMI.2025.3529294","DOIUrl":"10.1109/TMI.2025.3529294","url":null,"abstract":"Accurate and reliable annotation of anatomical landmarks in laparoscopic surgery remains a challenge due to varying degrees of landmark visibility and changing shapes of human tissues during a surgical procedure in videos. In this paper, we propose a knowledge-driven framework that integrates prior surgical expertise with visual data to address this problem. Inspired by visual reasoning knowledge of tool-anatomy interactions, our framework models a spatio-temporal graph to represent the static topology of tool and tissue and dynamic transitions of landmarks’ temporal behavior. By assigning explainable features of the surgical scene as node attributes in the graph, the surgical context is incorporated into the knowledge space. An attention-guided message passing mechanism across the graph dynamically adjusts the focus in different scenarios, enabling robust tracking of landmark states throughout the surgical process. Evaluations on the clinical dataset demonstrate the framework’s ability to effectively use the inductive bias of explainable features to label landmarks, showing its potential in tackling intricate surgical tasks with improved stability and reliability.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 5","pages":"2218-2229"},"PeriodicalIF":0.0,"publicationDate":"2025-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142981392","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Angle-Independent Blood Flow Velocity Measurement With Ultrasound Speckle Decorrelation Analysis","authors":"Yongchao Wang;Wenkai Chen;Yetao He;Jianbo Tang","doi":"10.1109/TMI.2025.3529033","DOIUrl":"10.1109/TMI.2025.3529033","url":null,"abstract":"Precise measurement of the blood flow velocity in major arteries is important for the assessment of circulation dysfunction but challenging when using a one-dimensional (1D) ultrasound transducer array. Current available ultrasound velocimetry methods are susceptible to the probe-to-vessel angle and require the vessels to be well-aligned within the imaging plane of the 1D transducer array. In this study, a novel angle-independent velocimetry (VT-vUS) based on the ultrasound speckle decorrelation analysis of the ultrasound field signal is proposed to measure the blood flow velocity using a conventional 1D ultrasound transducer array. We first introduced the principle and evaluated this technique with numerical simulation and phantom experiments, which demonstrated that VT-vUS can accurately reconstruct the velocity magnitude of blood flow at arbitrary probe-to-vessel angles for different preset flow speeds (up to ~2.5 m/s). Further, we applied VT-vUS to measure the pulsatile flow of the radial artery and carotid artery in a healthy volunteer. Results show that the absolute velocity profiles obtained with VT-vUS at different probe-to-vessel angles have high consistency and agree well with the absolute speed obtained with the color Doppler-corrected velocimetry throughout the cardiac cycle. With the ability to alleviate the dependency on probe-to-vessel angle, VT-vUS has the potential for circulation-related disease screening in clinical practices.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 5","pages":"2283-2294"},"PeriodicalIF":0.0,"publicationDate":"2025-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142981393","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"MIP-Enhanced Uncertainty-Aware Network for Fast 7T Time-of-Flight MRA Reconstruction","authors":"Kaicong Sun;Caohui Duan;Xin Lou;Dinggang Shen","doi":"10.1109/TMI.2025.3528402","DOIUrl":"10.1109/TMI.2025.3528402","url":null,"abstract":"Time-of-flight (TOF) magnetic resonance angiography (MRA) is the dominant non-contrast MR imaging method for visualizing intracranial vascular system. The employment of 7T MRI for TOF-MRA is of great interest due to its outstanding spatial resolution and vessel-tissue contrast. However, high-resolution 7T TOF-MRA is undesirably slow to acquire. Besides, due to complicated and thin structures of brain vessels, reliability of reconstructed vessels is of great importance. In this work, we propose an uncertainty-aware reconstruction model for accelerated 7T TOF-MRA, which combines the merits of deep unrolling and evidential deep learning, such that our model not only provides promising MRI reconstruction, but also supports uncertainty quantification within a single inference. Moreover, we propose a maximum intensity projection (MIP) loss for TOF-MRA reconstruction to improve the quality of MIP images. In the experiments, we have evaluated our model on a relatively large in-house multi-coil 7T TOF-MRA dataset extensively, showing promising superiority of our model compared to state-of-the-art models in terms of both TOF-MRA reconstruction and uncertainty quantification.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 5","pages":"2270-2282"},"PeriodicalIF":0.0,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142974783","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Michael Sommersperger;Philipp Matten;Tony Wang;Shervin Dehghani;Jonas Nienhaus;Hessam Roodaki;Wolfgang Drexler;Rainer A. Leitgeb;Tilman Schmoll;Nassir Navab
{"title":"Context-Aware Real-Time Semantic View Expansion of Intraoperative 4D OCT","authors":"Michael Sommersperger;Philipp Matten;Tony Wang;Shervin Dehghani;Jonas Nienhaus;Hessam Roodaki;Wolfgang Drexler;Rainer A. Leitgeb;Tilman Schmoll;Nassir Navab","doi":"10.1109/TMI.2025.3528742","DOIUrl":"10.1109/TMI.2025.3528742","url":null,"abstract":"Four-dimensional microscope-integrated optical coherence tomography enables volumetric imaging of tissue structures and tool-tissue interactions in ophthalmic surgery at interactive update rates. This enables surgeons to undertake particular surgical steps under four-dimensional optical coherence tomography (4D OCT) guidance. However, current 4D OCT systems are limited by their field of view and signal quality. Both are attributable to the emphasis on high volume acquisition rates, which is critical for smooth visual perception by the surgeon. Existing 3D volume mosaicing methods are developed in the context of diagnostic imaging and do not take dynamic surgical interactions and real-time processing into account. In this paper, we propose a novel volume mosaicing and visualization methodology that not only aims at leveraging the temporal information to overcome some of the current limitations and imaging artifacts of 4D OCT, but also is aware of the surgical context and dynamic instrument motion implicitly during registration and explicitly for visualization. We propose a rapid 4-degrees of freedom volume registration, integrating an innovative approach for volume mosaicing that takes temporal recency and semantic information into account for enhanced surgical visualization. Our experiments on 4D OCT datasets demonstrate high registration accuracy and illustrate the benefits for visualization by reducing imaging artifacts and dynamically expanding the surgical view.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 5","pages":"2256-2269"},"PeriodicalIF":0.0,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10838602","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142975081","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mengchu Wang;Yuhang He;Lin Peng;Xiang Song;Songlin Dong;Yihong Gong
{"title":"Cross-Domain Invariant Feature Absorption and Domain-Specific Feature Retention for Domain Incremental Chest X-Ray Classification","authors":"Mengchu Wang;Yuhang He;Lin Peng;Xiang Song;Songlin Dong;Yihong Gong","doi":"10.1109/TMI.2025.3525902","DOIUrl":"10.1109/TMI.2025.3525902","url":null,"abstract":"Chest X-ray (CXR) images have been widely adopted in clinical care and pathological diagnosis in recent years. Some advanced methods on CXR classification task achieve impressive performance by training the model statically. However, in the real clinical environment, the model needs to learn continually and this can be viewed as a domain incremental learning (DIL) problem. Due to large domain gaps, DIL is faced with catastrophic forgetting. Therefore, in this paper, we propose a Cross-domain invariant feature absorption and Domain-specific feature retention (CaD) framework. To be specific, we adopt a Cross-domain Invariant Feature Absorption (CIFA) module to learn the domain invariant knowledge and a Domain-Specific Feature Retention (DSFR) module to learn the domain-specific knowledge. The CIFA module contains the C(lass)-adapter and an absorbing strategy is used to fuse the common features among different domains. The DSFR module contains the D(omain)-adapter for each domain and it connects to the network in parallel independently to prevent forgetting. A multi-label contrastive loss (MLCL) is used in the training process and improves the class distinctiveness within each domain. We leverage publicly available large-scale datasets to simulate domain incremental learning scenarios, extensive experimental results substantiate the effectiveness of our proposed methods and it has reached state-of-the-art performance.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 5","pages":"2041-2055"},"PeriodicalIF":0.0,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142940138","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Fast 3D Breast Imaging With a Transmission-Based Microwave System","authors":"Pedram Mojabi;Jeremie Bourqui;Elise Fear","doi":"10.1109/TMI.2025.3527916","DOIUrl":"10.1109/TMI.2025.3527916","url":null,"abstract":"Microwave breast imaging has recently been explored for tumor detection, treatment monitoring, and estimating breast density. Only one prior work has presented quantitative three-dimensional (3D) breast imaging based on a full-wave inverse scattering approach applied to experimental data collected from human subjects; most other works rely on quantitative 2D images or qualitative reconstructions. This paper introduces a fast and efficient 3D quantitative reconstruction approach for microwave breast imaging without the need for prior information or iterative algorithms typically used in solving full-wave equations. The method assumes wave propagation in straight lines, similar to the ray tracing method used in ultrasound imaging, and formulates the algorithm based on this assumption. The algorithm is applied to data collected at multiple antennas over a wideband frequency range with a novel microwave transmission system. This system is designed to be in direct contact with the breast, eliminating the need for a matching medium. We experimentally demonstrate quantitative 3D permittivity reconstruction for graphite phantoms with various sizes and numbers of inclusions, comparing the results with available 3D CT scans of these phantoms. Next, we test this algorithm for 3D quantitative permittivity reconstruction in four healthy participants with different breast density categories and compare the images with their mammograms. Finally, the stability of the 3D permittivity reconstruction over three time points for the participants is demonstrated.","PeriodicalId":94033,"journal":{"name":"IEEE transactions on medical imaging","volume":"44 5","pages":"2206-2217"},"PeriodicalIF":0.0,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142940289","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}