IEEE Transactions on Computational Imaging最新文献_第8页

Provable Probabilistic Imaging Using Score-Based Generative Priors 利用基于分数的生成先验进行可证明的概率成像

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-08-23 DOI: 10.1109/TCI.2024.3449114

Yu Sun;Zihui Wu;Yifan Chen;Berthy T. Feng;Katherine L. Bouman

{"title":"Provable Probabilistic Imaging Using Score-Based Generative Priors","authors":"Yu Sun;Zihui Wu;Yifan Chen;Berthy T. Feng;Katherine L. Bouman","doi":"10.1109/TCI.2024.3449114","DOIUrl":"10.1109/TCI.2024.3449114","url":null,"abstract":"Estimating high-quality images while also quantifying their uncertainty are two desired features in an image reconstruction algorithm for solving ill-posed inverse problems. In this paper, we propose \u0000<italic>plug-and-play Monte Carlo (PMC)\u0000 as a principled framework for characterizing the space of possible solutions to a general inverse problem. PMC is able to incorporate expressive score-based generative priors for high-quality image reconstruction while also performing uncertainty quantification via posterior sampling. In particular, we develop two PMC algorithms that can be viewed as the sampling analogues of the traditional plug-and-play priors (PnP) and regularization by denoising (RED) algorithms. To improve the sampling efficiency, we introduce weighted annealing into these PMC algorithms, further developing two additional annealed PMC algorithms (APMC). We establish a theoretical analysis for characterizing the convergence behavior of PMC algorithms. Our analysis provides non-asymptotic stationarity guarantees in terms of the Fisher information, fully compatible with the joint presence of weighted annealing, potentially non-log-concave likelihoods, and imperfect score networks. We demonstrate the performance of the PMC algorithms on multiple representative inverse problems with both linear and nonlinear forward models. Experimental results show that PMC significantly improves reconstruction quality and enables high-fidelity uncertainty quantification.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1290-1305"},"PeriodicalIF":4.2,"publicationDate":"2024-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142177628","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Joint Reconstruction and Spatial Super-Resolution of Hyper-Spectral CTIS Images via Multi-Scale Refinement 通过多尺度细化实现超光谱 CTIS 图像的联合重建和空间超分辨率

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-08-20 DOI: 10.1109/TCI.2024.3446230

Mazen Mel;Alexander Gatto;Pietro Zanuttigh

{"title":"Joint Reconstruction and Spatial Super-Resolution of Hyper-Spectral CTIS Images via Multi-Scale Refinement","authors":"Mazen Mel;Alexander Gatto;Pietro Zanuttigh","doi":"10.1109/TCI.2024.3446230","DOIUrl":"10.1109/TCI.2024.3446230","url":null,"abstract":"The Computed Tomography Imaging Spectrometer (CTIS) is a snapshot imaging device that captures Hyper-Spectral images as two-dimensional compressed sensor measurements. Computational post-processing algorithms are later needed to recover the latent object cube. However, iterative algorithms typically used to solve this task require large computational resources and, furthermore, these approaches are very sensitive to the presumed system and noise models. In addition, the poor spatial resolution of the \u0000<inline-formula><tex-math>$0$</tex-math></inline-formula>\u0000th diffraction order image limits the usability of CTIS in favor of other snapshot spectrometers even though it enables higher spectral resolution. In this paper we introduce a learning-based computational model exploiting a reconstruction network with iterative refinement, that is able to recover high quality hyper-spectral images leveraging complementary spatio-spectral information scattered across the CTIS sensor image. We showcase the reconstruction capability of such model beyond the spatial resolution limit of the \u0000<inline-formula><tex-math>$0$</tex-math></inline-formula>\u0000th diffraction order image. Experimental results are shown both on synthetic data and on real datasets that we acquired using two different CTIS systems coupled with high spatial resolution ground truth hyper-spectral images. Furthermore, we introduce HSIRS, the largest dataset of its kind for joint spectral image reconstruction and semantic segmentation of food items with high quality manually annotated segmentation maps and we showcase how hyper-spectral data allows to efficiently tackle this task.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1449-1461"},"PeriodicalIF":4.2,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10640287","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142177629","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Hybrid Polarization Image Demosaicking Algorithm Based on Inter-Channel Correlation 基于信道间相关性的混合偏振图像去马赛克算法

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-08-15 DOI: 10.1109/TCI.2024.3443728

Yang Lu;Jiandong Tian;Yiming Su;Yidong Luo;Junchao Zhang;Chunhui Hao

{"title":"A Hybrid Polarization Image Demosaicking Algorithm Based on Inter-Channel Correlation","authors":"Yang Lu;Jiandong Tian;Yiming Su;Yidong Luo;Junchao Zhang;Chunhui Hao","doi":"10.1109/TCI.2024.3443728","DOIUrl":"10.1109/TCI.2024.3443728","url":null,"abstract":"Emerging \u0000<italic>monochrome and chromatic polarization filter array\u0000 (MPFA and CPFA) cameras require polarization demosaicking to obtain accurate polarization parameters. Polarization cameras sample the polarization intensity at each location of the pixels. A captured raw image must be converted to a full-channel polarization intensity image using the \u0000<italic>polarization demosaicking method\u0000 (PDM). However, due to sparse sampling between polarization channels, implementing MPFA and CPFA demosaicking has been challenging. This paper proposes a new hybrid polarization demosaicking algorithm that leverages polarization confidence-based refinement to exploit inter-channel polarization correlation. Additionally, we enhance texture correlation to utilize inter-channel texture correlation fully. Our three-stage PDM preserves both the polarization and texture information. We also introduce a metric computation method to handle the \u0000<inline-formula><tex-math>$pi$</tex-math></inline-formula>\u0000-ambiguity of the \u0000<italic>angle of line polarization\u0000 (AoLP). This approach mitigates inaccuracies and \u0000<inline-formula><tex-math>$pi$</tex-math></inline-formula>\u0000-ambiguity in existing methods when describing the quality of AoLP reconstruction. We extensively compare and conduct ablation experiments on synthetic datasets from MPFA and CPFA. Our method achieves competitive results compared to other state-of-the-art methods. Furthermore, we evaluate our proposal on real-world datasets to demonstrate its applicability in real-world, variable scenarios. Two application experiments (road detection and shape from polarization) show that our proposal can be applied to real-world applications.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1400-1413"},"PeriodicalIF":4.2,"publicationDate":"2024-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142177630","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Decoupling Image Deblurring Into Twofold: A Hierarchical Model for Defocus Deblurring 将图像去毛刺解耦为两部分：去焦点模糊的层次模型

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-08-15 DOI: 10.1109/TCI.2024.3443732

Pengwei Liang;Junjun Jiang;Xianming Liu;Jiayi Ma

{"title":"Decoupling Image Deblurring Into Twofold: A Hierarchical Model for Defocus Deblurring","authors":"Pengwei Liang;Junjun Jiang;Xianming Liu;Jiayi Ma","doi":"10.1109/TCI.2024.3443732","DOIUrl":"https://doi.org/10.1109/TCI.2024.3443732","url":null,"abstract":"Defocus deblurring, especially when facing spatially varying blur due to scene depth, remains a challenging problem. While recent advancements in network architectures have predominantly addressed high-frequency details, the importance of scene understanding for deblurring remains paramount. A crucial aspect of this understanding is \u0000<italic>contextual information\u0000, which captures vital high-level semantic cues essential for grasping the context and object outlines. Recognizing and effectively capitalizing on these cues can lead to substantial improvements in image recovery. With this foundation, we propose a novel method that integrates spatial details and contextual information, offering significant advancements in defocus deblurring. Consequently, we introduce a novel hierarchical model, built upon the capabilities of the Vision Transformer (ViT). This model seamlessly encodes both spatial details and contextual information, yielding a robust solution. In particular, our approach decouples the complex deblurring task into two distinct subtasks. The first is handled by a primary feature encoder that transforms blurred images into detailed representations. The second involves a contextual encoder that produces abstract and sharp representations from the primary ones. The combined outputs from these encoders are then merged by a decoder to reproduce the sharp target image. Our evaluation across multiple defocus deblurring datasets demonstrates that the proposed method achieves compelling performance.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1207-1220"},"PeriodicalIF":4.2,"publicationDate":"2024-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142099854","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An Inverse-Problem Approach to the Estimation of Despeckled and Deconvolved Images From Radio-Frequency Signals in Pulse-Echo Ultrasound 从脉冲回波超声波中的射频信号估算去斑和去卷积图像的逆问题方法

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-08-12 DOI: 10.1109/TCI.2024.3441234

Samuel Beuret;Adrien Besson;Akihiro Sugimoto;Jean-Philippe Thiran

{"title":"An Inverse-Problem Approach to the Estimation of Despeckled and Deconvolved Images From Radio-Frequency Signals in Pulse-Echo Ultrasound","authors":"Samuel Beuret;Adrien Besson;Akihiro Sugimoto;Jean-Philippe Thiran","doi":"10.1109/TCI.2024.3441234","DOIUrl":"https://doi.org/10.1109/TCI.2024.3441234","url":null,"abstract":"In recent years, there has been notable progress in the development of inverse problems for image reconstruction in pulse-echo ultrasound. Inverse problems are designed to circumvent the restrictions of delay-and-sum, such as limited image resolution and diffraction artifacts, especially when low amount of data are considered. However, the radio-frequency image or tissue reflectivity function that current inverse problems seek to estimate do not possess a structure that can be easily leveraged by a regularizer, in part due to their high dynamic range. The performance of inverse-problem image reconstruction is thus impeded. In contrast, despeckled images exhibit a more exploitable structure. Therefore, we first propose an inverse problem to recover a despeckled image from single-plane-wave radio-frequency echo signals, employing total-variation norm regularization. Then, we introduce an inverse problem to estimate the tissue reflectivity function from radio-frequency echo signals, factoring in the despeckled image obtained by the first problem into a spatially-varying reflectivity prior. We show with simulated, in-vitro, and in-vivo data that the proposed despeckled image estimation technique recovers images almost free of diffraction artifacts and improves contrast with respect to delay-and-sum and non-local means despeckling. Moreover, we show with in-vitro and in-vivo data that the proposed reflectivity estimation method reduces artifacts and improves contrast with respect to a state-of-the-art inverse problem positing a uniform prior. In particular, the proposed techniques could prove beneficial for imaging with ultra-portable transducers, since these devices are likely to be limited in the amount of data they can acquire and transmit.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1191-1206"},"PeriodicalIF":4.2,"publicationDate":"2024-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10634308","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142099726","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Learning Piecewise Planar Representation for RGB Guided Depth Super-Resolution 学习用于 RGB 引导深度超级分辨率的分片平面表示法

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-08-07 DOI: 10.1109/TCI.2024.3439990

Ruikang Xu;Mingde Yao;Yuanshen Guan;Zhiwei Xiong

{"title":"Learning Piecewise Planar Representation for RGB Guided Depth Super-Resolution","authors":"Ruikang Xu;Mingde Yao;Yuanshen Guan;Zhiwei Xiong","doi":"10.1109/TCI.2024.3439990","DOIUrl":"10.1109/TCI.2024.3439990","url":null,"abstract":"RGB guided depth super-resolution (GDSR) aims to reconstruct high-resolution (HR) depth images from low-resolution ones using HR RGB images as guidance, overcoming the resolution limitation of depth cameras. The main challenge in this task is how to effectively explore the HR information from RGB images while avoiding texture being over-transferred. To address this challenge, we propose a novel method for GSDR based on the piecewise planar representation in the 3D space, which naturally focuses on the geometry information of scenes without concerning the internal textures. Specifically, we design a plane-aware interaction module to effectively bridge the RGB and depth modalities and perform information interaction by taking piecewise planes as the intermediary. We also devise a plane-guided fusion module to further remove modality-inconsistent information. To mitigate the distribution gap between synthetic and real-world data, we propose a self-training adaptation strategy for the real-world deployment of our method. Comprehensive experimental results on multiple representative datasets demonstrate the superiority of our method over existing state-of-the-art GDSR methods.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1266-1279"},"PeriodicalIF":4.2,"publicationDate":"2024-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141941880","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Enhancing MRF Reconstruction: A Model-Based Deep Learning Approach Leveraging Learned Sparsity and Physics Priors 增强 MRF 重构：利用学习稀疏性和物理先验的基于模型的深度学习方法

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-08-07 DOI: 10.1109/TCI.2024.3440008

Peng Li;Yue Hu

{"title":"Enhancing MRF Reconstruction: A Model-Based Deep Learning Approach Leveraging Learned Sparsity and Physics Priors","authors":"Peng Li;Yue Hu","doi":"10.1109/TCI.2024.3440008","DOIUrl":"10.1109/TCI.2024.3440008","url":null,"abstract":"Deep learning has shown great promise in improving the speed and accuracy of parameter map estimation in magnetic resonance fingerprinting (MRF). However, many existing methods rely on physics-free networks, leading to a staged processing strategy. This strategy involves the initial reconstruction of acquired non-Cartesian undersampled measurements, followed by subsequent parameter map estimation. Unfortunately, such a staged processing strategy may lead to partial information loss and limit the eventual accuracy of parameter imaging. To overcome these challenges, in this paper, we propose a novel model-based deep learning approach that directly reconstructs parameter maps from non-Cartesian undersampled measurements. Specifically, our approach first incorporates MRF imaging physics priors and data correlation constraints into a unified reconstruction model. The proposed model-based network, named LS-MRF-Net, is then defined by unrolling the iterative procedures of the reconstruction model into a deep neural network. Notably, a learned sparsity layer is proposed to exploit the optimal transform domain for sparse representation of high-dimensional MRF data. Additionally, we incorporate a mapping layer and a Bloch response dynamic layer to seamlessly integrate the MRF imaging physics priors into the network. Experimental results on both simulated and \u0000<italic>in vivo\u0000 datasets demonstrate that the proposed method can significantly reduce computational time while enhancing MRF reconstruction performance.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1221-1234"},"PeriodicalIF":4.2,"publicationDate":"2024-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141941879","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

NAS Powered Deep Image Prior for Electrical Impedance Tomography NAS 驱动的电阻抗断层扫描深度图像先验

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-08-07 DOI: 10.1109/TCI.2024.3440063

Haoyuan Xia;Qianxue Shan;Junwu Wang;Dong Liu

引用次数: 0

Real-Time Model-Based Quantitative Ultrasound and Radar 基于模型的实时定量超声波和雷达

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-08-02 DOI: 10.1109/TCI.2024.3436537

Tom Sharon;Yonina C. Eldar

引用次数: 0

CMEFusion: Cross-Modal Enhancement and Fusion of FIR and Visible Images CMEFusion：傅立叶变换和可见光图像的跨模态增强与融合

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-08-02 DOI: 10.1109/TCI.2024.3436716

Xi Tong;Xing Luo;Jiangxin Yang;Yanpeng Cao

{"title":"CMEFusion: Cross-Modal Enhancement and Fusion of FIR and Visible Images","authors":"Xi Tong;Xing Luo;Jiangxin Yang;Yanpeng Cao","doi":"10.1109/TCI.2024.3436716","DOIUrl":"10.1109/TCI.2024.3436716","url":null,"abstract":"The fusion of far infrared (FIR) and visible images aims to generate a high-quality composite image that contains salient structures and abundant texture details for human visual perception. However, the existing fusion methods typically fall short of utilizing complementary source image characteristics to boost the features extracted from degraded visible or FIR images, thus they cannot generate satisfactory fusion results in adverse lighting or weather conditions. In this paper, we propose a novel Cross-Modal multispectral image Enhancement and Fusion framework (CMEFusion), which adaptively enhances both FIR and visible inputs by leveraging complementary cross-modal features to further facilitate multispectral feature aggregation. Specifically, we first present a new cross-modal image enhancement sub-network (CMIENet), which is built on a CNN-Transformer hybrid architecture to perform the complementary exchange of local-salient and global-contextual features extracted from FIR and visible modalities, respectively. Then, we design a gradient-content differential fusion sub-network (GCDFNet) to progressively integrate decoupled gradient and content information via modified central difference convolution. Finally, we present a comprehensive joint enhancement-fusion multi-term loss function to drive the model to narrow the optimization gap between the above-mentioned two sub-networks based on the self-supervised aspects of exposure, color, structure, and intensity. In this manner, the proposed CMEFusion model facilitates better-performing visible and FIR image fusion in an end-to-end way, achieving enhanced visual quality with more natural and realistic appearances. Extensive experiments validate that CMEFusion surpasses state-of-the-art image fusion algorithms, as evidenced by superior performance in both visual quality and quantitative evaluations.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1331-1345"},"PeriodicalIF":4.2,"publicationDate":"2024-08-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141886639","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0