IEEE Transactions on Computational Imaging最新文献

{"title":"Fixed-Point Convergence of Multi-Block PnP ADMM and Its Application to Hyperspectral Image Restoration","authors":"Weijie Liang;Zhihui Tu;Jian Lu;Kai Tu;Michael K. Ng;Chen Xu","doi":"10.1109/TCI.2024.3485467","DOIUrl":"https://doi.org/10.1109/TCI.2024.3485467","url":null,"abstract":"Coupling methods of integrating multiple priors have emerged as a pivotal research focus in hyperspectral image (HSI) restoration. Among these methods, the Plug-and-Play (PnP) framework stands out and pioneers a novel coupling approach, enabling flexible integration of diverse methods into model-based approaches. However, the current convergence analyses of the PnP framework are highly unexplored, as they are limited to 2-block composite optimization problems, failing to meet the need of coupling modeling for incorporating multiple priors. This paper focuses on the convergence analysis of PnP-based algorithms for multi-block composite optimization problems. In this work, under the PnP framework and utilizing the alternating direction method of multipliers (ADMM) of the continuation scheme, we propose a unified multi-block PnP ADMM algorithm framework for HSI restoration. Inspired by the fixed-point convergence theory of the 2-block PnP ADMM, we establish a similar fixed-point convergence guarantee for the multi-block PnP ADMM with extended condition and provide a feasible parameter tuning methodology. Based on this framework, we design an effective mixed noise removal algorithm incorporating global, nonlocal and deep priors. Extensive experiments validate the algorithm's superiority and competitiveness.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1571-1587"},"PeriodicalIF":4.2,"publicationDate":"2024-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142636307","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"CalibFPA: A Focal Plane Array Imaging System Based on Online Deep-Learning Calibration","authors":"Alper Güngör;M. Umut Bahceci;Yasin Ergen;Ahmet Sözak;O. Oner Ekiz;Tolga Yelboga;Tolga Çukur","doi":"10.1109/TCI.2024.3477312","DOIUrl":"https://doi.org/10.1109/TCI.2024.3477312","url":null,"abstract":"Compressive focal plane arrays (FPA) enable cost-effective high-resolution (HR) imaging by acquisition of several multiplexed measurements on a low-resolution (LR) sensor. Multiplexed encoding of the visual scene is often attained via electronically controllable spatial light modulators (SLM). To capture system non-idealities such as optical aberrations, a system matrix is measured via additional offline scans, where the system response is recorded for a point source at each spatial location on the imaging grid. An HR image can then be reconstructed by solving an inverse problem that involves encoded measurements and the calibration matrix. However, this offline calibration framework faces limitations due to challenges in encoding single HR grid locations with a fixed coded aperture, lengthy calibration scans repeated to account for system drifts, and computational burden of reconstructions based on dense system matrices. Here, we propose a novel compressive FPA system based on online deep-learning calibration of multiplexed LR measurements (CalibFPA). To acquire multiplexed measurements, we devise an optical setup where a piezo-stage locomotes a pre-printed fixed coded aperture. We introduce a physics-driven deep-learning method to correct for the influences of optical aberrations in multiplexed measurements without the need for offline calibration scans. The corrected measurement matrix is of block-diagonal form, so it can be processed efficiently to recover HR images with a user-preferred reconstruction algorithm including least-squares, plug-and-play, or unrolled techniques. On simulated and experimental datasets, we demonstrate that CalibFPA outperforms state-of-the-art compressive FPA methods. We also report analyses to validate the design elements in CalibFPA and assess computational complexity.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1650-1663"},"PeriodicalIF":4.2,"publicationDate":"2024-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142736588","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Deep Motion Regularizer for Video Snapshot Compressive Imaging","authors":"Zan Chen;Ran Li;Yongqiang Li;Yuanjing Feng;Xingsong Hou;Xueming Qian","doi":"10.1109/TCI.2024.3477262","DOIUrl":"https://doi.org/10.1109/TCI.2024.3477262","url":null,"abstract":"Video snapshot compressive imaging (SCI) samples 3D high-speed video frames with temporally varying spatial modulation and compresses them into a single 2D measurement, and the SCI reconstruction algorithm aims to recover the original high-speed frames from the measurement. However, conventional video SCI systems encounter challenges when raising the frame rate of the reconstructed video. To achieve higher frame rates, the modulation mask's rate must be increased, which in turn leads to an increase in the associated hardware expenses. In this paper, we propose a deep unfolding-based reconstruction framework with optical flow for video SCI. The framework recovers both observed and unobserved frames from measurements, resulting in increased video frame rate. To estimate the optical flow, we transform the video features of the network into optical flow features during the iteration. Then, we design a deep denoiser and an optical flow-based motion regularizer combined with the voxels of coarse reconstructed frames to update the observed and unobserved frames. To improve the performance, we employ group convolution in the network and fuse the optical flow information from different phases to reduce the information loss. We further extend the proposed deep unfolding framework to the reconstruction of color SCI videos. Extensive experiments on benchmark data and real data prove that our proposed method has state-of-the-art reconstruction performance and visual effects.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1519-1532"},"PeriodicalIF":4.2,"publicationDate":"2024-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142524174","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Provable Preconditioned Plug-and-Play Approach for Compressed Sensing MRI Reconstruction","authors":"Tao Hong;Xiaojian Xu;Jason Hu;Jeffrey A. Fessler","doi":"10.1109/TCI.2024.3477329","DOIUrl":"https://doi.org/10.1109/TCI.2024.3477329","url":null,"abstract":"Model-based methods play a key role in the reconstruction of compressed sensing (CS) MRI. Finding an effective prior to describe the statistical distribution of the image family of interest is crucial for model-based methods. Plug-and-play (PnP) is a general framework that uses denoising algorithms as the prior or regularizer. Recent work showed that PnP methods with denoisers based on pretrained convolutional neural networks outperform other classical regularizers in CS MRI reconstruction. However, the numerical solvers for PnP can be slow for CS MRI reconstruction. This paper proposes a preconditioned PnP (\u0000<inline-formula><tex-math>$text{P}^{2}$</tex-math></inline-formula>\u0000nP) method to accelerate the convergence speed. Moreover, we provide proofs of the fixed-point convergence of the \u0000<inline-formula><tex-math>$text{P}^{2}$</tex-math></inline-formula>\u0000nP iterates. Numerical experiments on CS MRI reconstruction with non-Cartesian sampling trajectories illustrate the effectiveness and efficiency of the \u0000<inline-formula><tex-math>$text{P}^{2}$</tex-math></inline-formula>\u0000nP approach.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1476-1488"},"PeriodicalIF":4.2,"publicationDate":"2024-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142452742","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Quantum Denoising-Based Resolution Enhancement Framework for 250-MHz and 500-MHz Quantitative Acoustic Microscopy","authors":"Sayantan Dutta;Jonathan Mamou","doi":"10.1109/TCI.2024.3473312","DOIUrl":"https://doi.org/10.1109/TCI.2024.3473312","url":null,"abstract":"Quantitative acoustic microscopy (QAM) forms two-dimensional (2D) quantitative maps of acoustic properties of thin tissue sections at a microscopic scale (\u0000<inline-formula><tex-math>$&lt; 8; mu$</tex-math></inline-formula>\u0000m) using very-high-frequency (i.e., \u0000<inline-formula><tex-math>$&gt;$</tex-math></inline-formula>\u0000 200 MHz) ultrasonic excitation. Our custom-made QAM systems employ a 250-MHz or a 500-MHz single-element transducer to produce 2D maps with theoretical spatial resolutions smaller than 8 \u0000<inline-formula><tex-math>$mu$</tex-math></inline-formula>\u0000m and 4 \u0000<inline-formula><tex-math>$mu$</tex-math></inline-formula>\u0000m, respectively. Even with the utilization of these state-of-the-art QAM instruments, spatial resolution still proves insufficient for certain clinical studies. However, designing a QAM system yielding finer resolution (i.e., using a higher-frequency transducer) is expensive and requires expert users. This work proposes a scheme to enhance the spatial resolution of the 2D QAM maps by exploiting an off-the-shelf quantum-based adaptive denoiser (DeQuIP), leveraging the principles of quantum many-body theory. Drawing upon the recent advancement in regularization-by-denoising (RED) for image restoration, we impose this external DeQuIP denoiser as a RED-prior coupled with an analytical solution to address the degradation operators in solving the QAM super-resolution problem. The efficiency of our proposed scheme is demonstrated by improving the resolution of experimental 2D acoustic-impedance maps (2DZMs) generated from data acquired using the 250-MHz and 500-MHz QAM systems. Our scheme demonstrates superior performance in recovering finer and subtle details with enhanced spatial resolution when applied to 2DZMs. For example, a spatial resolution improvement of 40% was achieved when applied to 2DZMs at 250-MHz, outperforming two other state-of-the-art methods, which only yielded 23–32% improvement. These observations highlight the efficacy of the proposed RED scheme.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1489-1504"},"PeriodicalIF":4.2,"publicationDate":"2024-10-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142518159","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"TMSST-RID: A Target-Oriented SAR-ISAR Imaging Method Based on Synchrosqueezing","authors":"Zhifeng Xie;Lei Cui;Qingyuan Shen;Xiaoqing Wang;Peiqing Yang;Xingyi Su;Haifeng Huang","doi":"10.1109/TCI.2024.3468032","DOIUrl":"https://doi.org/10.1109/TCI.2024.3468032","url":null,"abstract":"Synthetic aperture radar (SAR) is an important high-resolution radar system for target surveillance. However, target motion causes a time-varying Doppler frequency in echo and defocused SAR images. To obtain a focused target image, the inverse SAR synthetic aperture radar (ISAR) algorithm is applied: the image is inverted back to the echo and focused using the ISAR imaging algorithm. Clutter significantly impacts the target focusing effect during the SAR-ISAR imaging process. To improve the SAR-ISAR imaging quality of ship targets in the presence of severe sea clutter interference, this article proposes a range instantaneous Doppler method based on the low-rank sparse decomposition and target-oriented multisynchrosqueezing transform (TMSST-RID), which can enhance the target while suppressing clutter. The target area is separated from the clutter area using the low-rank sparse decomposition method. Then, in the time–frequency (TF) analysis, our proposed TMSST method utilizes the prior of the target area to improve the signal-to-clutter ratio (SCR) of the SAR/ISAR image. Compared with the traditional TF transformation, the proposed TMSST enhances the target by sharpening the TF target representation while simultaneously avoiding clutter enhancement. The superiority of this SAR/ISAR imaging method is demonstrated in a performance evaluation using simulated and real radar data for ship targets in sea clutter.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1425-1438"},"PeriodicalIF":4.2,"publicationDate":"2024-09-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142383484","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Accelerated Wirtinger Flow With Score-Based Image Priors for Holographic Phase Retrieval in Poisson-Gaussian Noise Conditions","authors":"Zongyu Li;Jason Hu;Xiaojian Xu;Liyue Shen;Jeffrey A. Fessler","doi":"10.1109/TCI.2024.3458418","DOIUrl":"https://doi.org/10.1109/TCI.2024.3458418","url":null,"abstract":"Phase retrieval (PR) is a crucial problem in many imaging applications. This study focuses on holographic phase retrieval in situations where the measurements are degraded by a combination of Poisson and Gaussian noise, as commonly occurs in optical imaging systems. We propose a new algorithm called “AWFS” that uses accelerated Wirtinger flow (AWF) with a learned score function as a generative prior. Specifically, we formulate the PR problem as an optimization problem that incorporates both data fidelity and regularization terms. We calculate the gradient of the log-likelihood function for PR and determine its corresponding Lipschitz constant. Additionally, we introduce a generative prior in our regularization framework by using score matching to capture information about the gradient of image prior distributions. We provide theoretical analysis that establishes a critical-point convergence guarantee for one version of the proposed algorithm. The results of our simulation experiments on three different datasets show the following. 1) By using the PG likelihood model, a practical version of the proposed algorithm improves reconstruction compared to algorithms based solely on Gaussian or Poisson likelihoods. 2) The proposed score-based image prior method leads to better reconstruction quality than a method based on denoising diffusion probabilistic model (DDPM), as well as a plug-and-play alternating direction method of multipliers (PnP-ADMM) and regularization by denoising (RED).","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1384-1399"},"PeriodicalIF":4.2,"publicationDate":"2024-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142328360","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Spectral Memory-Enhanced Network With Local Non-Local and Low-Rank Priors for Hyperspectral Image Compressive Imaging","authors":"Yangke Ying;Jin Wang;Yunhui Shi;Nam Ling;Baocai Yin","doi":"10.1109/TCI.2024.3468615","DOIUrl":"https://doi.org/10.1109/TCI.2024.3468615","url":null,"abstract":"The hyperspectral image (HSI) compressive imaging field has experienced significant progress in recent years, especially with the emergence of deep unfolding networks (DUNs), which have demonstrated remarkable advancements in reconstruction performance. However, these methods still face several challenges. Firstly, HSI data carries crucial prior knowledge in the feature space, and effectively leveraging these priors is essential for achieving high-quality HSI reconstruction. Existing methods either neglect the utilization of prior information or incorporate network modules designed based on prior information in a rudimentary manner, thereby limiting the overall reconstruction potential of these models. Secondly, the transformation between the data and feature domains poses a significant challenge for DUNs, leading to the loss of feature information across different stages. Existing methods fall short in adequately considering spectral characteristics when utilizing inter-stage information, resulting in inefficient transmission of feature information. In this paper, we introduce a novel deep unfolding network architecture that integrates local non-local and low-rank priors with spectral memory enhancement for precise HSI data reconstruction. Specifically, we design innovative modules for local non-local and low-rank priors to enrich the network's feature representation capability, fully exploiting the prior information of HSI data in the feature space. These designs also help the overall framework achieve superior reconstruction results with fewer parameters. Moreover, we extensively consider the spectral correlation characteristics of HSI data and devise a spectral memory enhancement network module to mitigate inter-stage feature information loss. Extensive experiments further demonstrate the superiority of our approach.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1664-1679"},"PeriodicalIF":4.2,"publicationDate":"2024-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142753892","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Fast Non-Line-of-Sight Imaging With Hybrid Super-Resolution Network Over 18 m","authors":"Leping Xiao;Jianyu Wang;Yi Wang;Ziyu Zhan;Zuoqiang Shi;Lingyun Qiu;Xing Fu","doi":"10.1109/TCI.2024.3463964","DOIUrl":"https://doi.org/10.1109/TCI.2024.3463964","url":null,"abstract":"Non-line-of-sight (NLOS) imaging technique aims at visualizing hidden objects from light of multiple reflections. For most existing methods, densely raster-scanned transients with long exposure time are routinely used, while approaches employing fewer points are confronted with a trade-off between the computation time and the image quality, both of which hinder the practical implementation of fast NLOS imaging. In this paper, we propose a hybrid super-resolution pipeline for image reconstruction and quality enhancement with only 8×8 scanning points. Besides, we implement a non-coaxial transceiver configuration and illustrate the first auto-calibration method for out-of-lab NLOS configuration, which costs only 40 s and performs well at a distance of 18.69 m. Results on both experimental data and public dataset indicate that the proposed method exhibits strong generalization capabilities, yielding faithful reconstructions with the resolution of 256×256 under different noise models. Furthermore, we demonstrate the importance of matching the noise model with the experimental dataset. We believe our approach shows great promise to NLOS imaging acceleration with lower acquisition, calibration and computation time.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1439-1448"},"PeriodicalIF":4.2,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142430859","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"3D Helical CT Reconstruction With a Memory Efficient Learned Primal-Dual Architecture","authors":"Jevgenija Rudzusika;Buda Bajić;Thomas Koehler;Ozan Öktem","doi":"10.1109/TCI.2024.3463485","DOIUrl":"10.1109/TCI.2024.3463485","url":null,"abstract":"Deep learning based computed tomography (CT) reconstruction has demonstrated outstanding performance on simulated 2D low-dose CT data. This applies in particular to domain adapted neural networks, which incorporate a handcrafted physics model for CT imaging. Empirical evidence shows that employing such architectures reduces the demand for training data and improves upon generalization. However, their training requires large computational resources that quickly become prohibitive in 3D helical CT, which is the most common acquisition geometry used for medical imaging. This paper modifies a domain adapted neural network architecture, the Learned Primal-Dual (LPD), so that it can be trained and applied to reconstruction in this setting. The main challenge is to reduce the GPU memory requirements during the training, while keeping the computational time within practical limits. Furthermore, clinical data also comes with other challenges not accounted for in simulations, like errors in flux measurement, resolution mismatch and, most importantly, the absence of the real ground truth. To the best of our knowledge, this work is the first to apply an unrolled deep learning architecture for reconstruction on full-sized clinical data, like those in the Low dose CT image and projection data set (LDCT).","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1414-1424"},"PeriodicalIF":4.2,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142249426","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}