IEEE Transactions on Computational Imaging最新文献

{"title":"PINQI: An End-to-End Physics-Informed Approach to Learned Quantitative MRI Reconstruction","authors":"Felix F. Zimmermann;Christoph Kolbitsch;Patrick Schuenke;Andreas Kofler","doi":"10.1109/TCI.2024.3388869","DOIUrl":"10.1109/TCI.2024.3388869","url":null,"abstract":"Quantitative Magnetic Resonance Imaging (qMRI) enables the reproducible measurement of biophysical parameters in tissue. The challenge lies in solving a nonlinear, ill-posed inverse problem to obtain the desired tissue parameter maps from acquired raw data. While various learned and non-learned approaches have been proposed, the existing learned methods fail to fully exploit the prior knowledge about the underlying MR physics, i.e. the signal model and the acquisition model. In this paper, we propose PINQI, a novel qMRI reconstruction method that integrates the knowledge about the signal, acquisition model, and learned regularization into a single end-to-end trainable neural network. Our approach is based on unrolled alternating optimization, utilizing differentiable optimization blocks to solve inner linear and non-linear optimization tasks, as well as convolutional layers for regularization of the intermediate qualitative images and parameter maps. This design enables PINQI to leverage the advantages of both the signal model and learned regularization. We evaluate the performance of our proposed network by comparing it with recently published approaches in the context of highly undersampled \u0000<inline-formula><tex-math>$T_{1}$</tex-math></inline-formula>\u0000-mapping, using both a simulated brain dataset, as well as real scanner data acquired from a physical phantom and in-vivo data from healthy volunteers. The results demonstrate the superiority of our proposed solution over existing methods and highlight the effectiveness of our method in real-world scenarios.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"628-639"},"PeriodicalIF":5.4,"publicationDate":"2024-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10499888","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140593484","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Fourier-Domain Inversion for the Modulo Radon Transform","authors":"Matthias Beckmann;Ayush Bhandari;Meira Iske","doi":"10.1109/TCI.2024.3388871","DOIUrl":"10.1109/TCI.2024.3388871","url":null,"abstract":"Inspired by the multiple-exposure fusion approach in computational photography, recently, several practitioners have explored the idea of high dynamic range (HDR) X-ray imaging and tomography. While establishing promising results, these approaches inherit the limitations of multiple-exposure fusion strategy. To overcome these disadvantages, the modulo Radon transform (MRT) has been proposed. The MRT is based on a co-design of hardware and algorithms. In the hardware step, Radon transform projections are folded using modulo non-linearities. Thereon, recovery is performed by algorithmically inverting the folding, thus enabling a single-shot, HDR approach to tomography. The first steps in this topic established rigorous mathematical treatment to the problem of reconstruction from folded projections. This paper takes a step forward by proposing a new, Fourier domain recovery algorithm that is backed by mathematical guarantees. The advantages include recovery at lower sampling rates while being agnostic to modulo threshold, lower computational complexity and empirical robustness to system noise. Beyond numerical simulations, we use prototype modulo ADC based hardware experiments to validate our claims. In particular, we report image recovery based on hardware measurements up to 10 times larger than the sensor's dynamic range while benefiting with lower quantization noise (\u0000<inline-formula><tex-math>$sim$</tex-math></inline-formula>\u000012 dB).","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"653-665"},"PeriodicalIF":5.4,"publicationDate":"2024-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140593768","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Nonlocal Tensor Decomposition With Joint Low Rankness and Smoothness for Spectral CT Image Reconstruction","authors":"Chunyan Liu;Sui Li;Dianlin Hu;Jianjun Wang;Wenjin Qin;Chen Liu;Peng Zhang","doi":"10.1109/TCI.2024.3384812","DOIUrl":"10.1109/TCI.2024.3384812","url":null,"abstract":"Spectral computed tomography (CT) is a medical imaging technology that utilizes the measurement of X-ray energy absorption in human tissue to obtain image information. It can provide more accurate and detailed image information, thereby improving the accuracy of diagnosis. However, the process of spectral CT imaging is usually accompanied by a large amount of radiation and noise, which makes it difficult to obtain high-quality spectral CT image. Therefore, this paper constructs a basic third-order tensor unit based on the self-similarity of patches in the spatial domain and spectral domain while proposing nonlocal spectral CT image reconstruction methods to obtain high-quality spectral CT image. Specifically, the algorithm decomposes the recombination tensor into a low-rank tensor and a sparse tensor, which are applied by weighted tensor nuclear norm (WTNN) and weighted tensor total variation (WTTV) norm to improve the reconstruction quality, respectively. In order to further improve algorithm performance, this paper also uses weighted tensor correlated total variation regularization(WTCTV) to simultaneously characterize the low rankness and smoothness of low-rank tensor, while the sparse tensor uses weighted tensor total variation regularization (WTTV) to represent the piecewise smooth structure of the spatial domain and the similarity between pixels and adjacent frames in the spectral domain. Hence, the proposed models can effectively provide faithful underlying information of spectral CT image while maintaining spatial structure. In addition, this paper uses the Alternating Direction Method of Multipliers(ADMM) to optimize the proposed spectral CT image reconstruction models. To verify the performance of the proposed algorithms, we conducted a large number of experiments on numerical phantom and clinic patient data. The experimental results indicate that incorporating weighted regularization outperforms the results without weighted regularization, and nonlocal similarity can achieve better results than that without nonlocal similarity. Compared with existing popular algorithms, the proposed models significantly reduce running time and improve the quality of spectral CT image, thereby assisting doctors in more accurate diagnosis and treatment of diseases.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"613-627"},"PeriodicalIF":5.4,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140593186","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Frequency-Spatial Domain Feature Fusion for Spectral Super-Resolution","authors":"Lishan Tan;Renwei Dian;Shutao Li;Jinyang Liu","doi":"10.1109/TCI.2024.3384811","DOIUrl":"https://doi.org/10.1109/TCI.2024.3384811","url":null,"abstract":"The purpose of spectral super-resolution (SSR) is to reconstruct hyperspectral image (HSI) from RGB image, which significantly reduces the difficulty of acquiring HSI. Most existing SSR methods adopt convolutional neural networks (CNNs) as the basic framework. The capability of CNNs to capture global context is limited, which constrains the performance of SSR. In this paper, we propose a novel frequency-spatial domain feature fusion network (FSDFF) for SSR, which simultaneously learns and fuses the frequency and spatial domain features of HSI. Frequency domain features can reflect the global information of image, which can be used to obtain the global context of HSI, thereby alleviating the limitations of CNNs in capturing global context. Spatial domain features contain abundant local structural information, which is beneficial for preserving spatial details in the SSR task. The mutual fusion of the two can better model the interrelationship between HSI and RGB image, thereby achieving better SSR performance. In FSDFF, we design a frequency domain feature learning branch (FDFL) and a spatial domain feature learning branch (SDFL) to learn the frequency and spatial domain features of HSI. Furthermore, a cross-domain feature fusion module (CDFF) is designed to facilitate the complementary fusion of the two types of features. The experimental results on two public datasets indicate that FSDFF has achieved state-of-the-art performance.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"589-599"},"PeriodicalIF":5.4,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140555973","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Computational Imaging for Machine Perception: Transferring Semantic Segmentation Beyond Aberrations","authors":"Qi Jiang;Hao Shi;Shaohua Gao;Jiaming Zhang;Kailun Yang;Lei Sun;Huajian Ni;Kaiwei Wang","doi":"10.1109/TCI.2024.3380363","DOIUrl":"https://doi.org/10.1109/TCI.2024.3380363","url":null,"abstract":"Semantic scene understanding with Minimalist Optical Systems (MOS) in mobile and wearable applications remains a challenge due to the corrupted imaging quality induced by optical aberrations. However, previous works only focus on improving the subjective imaging quality through the Computational Imaging (CI) technique, ignoring the feasibility of advancing semantic segmentation. In this paper, we pioneer the investigation of Semantic Segmentation under Optical Aberrations (SSOA) with MOS. To benchmark SSOA, we construct \u0000<italic>Virtual Prototype Lens (VPL)</i>\u0000 groups through optical simulation, generating \u0000<italic>Cityscapes-ab</i>\u0000 and \u0000<italic>KITTI-360-ab</i>\u0000 datasets under different behaviors and levels of aberrations. We look into SSOA via an unsupervised domain adaptation perspective to address the scarcity of labeled aberration data in real-world scenarios. Further, we propose \u0000<italic>Computational Imaging Assisted Domain Adaptation (CIADA)</i>\u0000 to leverage prior knowledge of CI for robust performance in SSOA. Based on our benchmark, we conduct experiments on the robustness of classical segmenters against aberrations. In addition, extensive evaluations of possible solutions to SSOA reveal that CIADA achieves superior performance under all aberration distributions, bridging the gap between computational imaging and downstream applications for MOS.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"535-548"},"PeriodicalIF":5.4,"publicationDate":"2024-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140533399","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Differentiable Deflectometric Eye Tracking","authors":"Tianfu Wang;Jiazhang Wang;Nathan Matsuda;Oliver Cossairt;Florian Willomitzer","doi":"10.1109/TCI.2024.3382494","DOIUrl":"10.1109/TCI.2024.3382494","url":null,"abstract":"Eye tracking is an important tool in many scientific and commercial domains. State-of-the-art eye tracking methods are either reflection-based and track reflections of sparse point light sources, or image-based and exploit 2D features of the acquired eye image. In this work, we attempt to significantly improve reflection-based methods by utilizing pixel-dense deflectometric surface measurements in combination with optimization-based inverse rendering algorithms. Utilizing the known geometry of our deflectometric setup, we develop a differentiable rendering pipeline based on PyTorch3D that simulates a virtual eye under screen illumination. Eventually, we exploit the image-screen-correspondence information from the captured measurements to find the eye's \u0000<italic>rotation</i>\u0000, \u0000<italic>translation</i>\u0000, and \u0000<italic>shape</i>\u0000 parameters with our renderer via gradient descent. We demonstrate real-world experiments with evaluated mean relative gaze errors below \u0000<inline-formula><tex-math>$0.45 ^{circ }$</tex-math></inline-formula>\u0000 at a precision better than \u0000<inline-formula><tex-math>$0.11 ^{circ }$</tex-math></inline-formula>\u0000. Moreover, we show an improvement of 6X over a representative reflection-based state-of-the-art method in simulation. In addition, we demonstrate a special variant of our method that does not require a specific pattern and can work with arbitrary image or video content from every screen (e.g., in a VR headset).","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"888-898"},"PeriodicalIF":5.4,"publicationDate":"2024-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140593185","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"HoloFormer: Contrastive Regularization Based Transformer for Holographic Image Reconstruction","authors":"Ziqi Bai;Xianming Liu;Cheng Guo;Kui Jiang;Junjun Jiang;Xiangyang Ji","doi":"10.1109/TCI.2024.3384809","DOIUrl":"https://doi.org/10.1109/TCI.2024.3384809","url":null,"abstract":"Deep learning has emerged as a prominent technique in the field of holographic imaging, owing to its rapidity and high performance. Prevailing deep neural networks employed for holographic image reconstruction predominantly rely on convolutional neural networks (CNNs). While CNNs have yielded impressive results, their intrinsic limitations, characterized by a constrained local receptive field and uniform representation, pose challenges in harnessing spatial texture similarities inherent in holographic images. To address this issue, we propose a novel hierarchical framework based on self-attention mechanism for digital holographic reconstruction, termed HoloFormer. Specifically, we adopt a window-based transformer block as the backbone, significantly reducing computational costs. In the encoder, a pyramid-like hierarchical structure enables the learning of feature map representations at different scales. In the decoder, a dual-branch design ensures that the real and imaginary parts of the complex amplitude do not exhibit cross-talk with each other. During the training phase, we incorporate contrastive regularization to maximize the utilization of mutual information. Overall, our experiments demonstrate that HoloFormer achieves superior reconstruction results compared to previous CNN-based architectures. This progress further propels the development of deep learning-based holographic imaging, particularly in lensless microscopy applications.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"560-573"},"PeriodicalIF":5.4,"publicationDate":"2024-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140546566","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"DD-DCSR: Image Denoising for Low-Dose CT via Dual-Dictionary Deep Convolutional Sparse Representation","authors":"Shu Li;Yi Liu;Rongbiao Yan;Haowen Zhang;Shubin Wang;Ting Ding;Zhiguo Gui","doi":"10.1109/TCI.2024.3408091","DOIUrl":"10.1109/TCI.2024.3408091","url":null,"abstract":"Most of the existing low-dose computed tomography (LDCT) denoising algorithms, based on convolutional neural networks, are not interpretable enough due to a lack of mathematical basis. In the process of image denoising, the sparse representation based on a single dictionary cannot restore the texture details of the image perfectly. To solve these problems, we propose a Dual-Dictionary Convolutional Sparse Representation (DD-CSR) method and construct a Dual-Dictionary Deep Convolutional Sparse Representation network (DD-DCSR) to unfold the model iteratively. The modules in the network correspond to the model one by one. In the proposed DD-CSR, the high-frequency information is extracted by Local Total Variation (LTV), and then two different learnable convolutional dictionaries are used to sparsely represent the LDCT image and its high-frequency map. To improve the robustness of the model, the adaptive coefficient is introduced into the convolutional dictionary of LDCT images, which allows the image to be represented by fewer convolutional dictionary atoms and reduces the number of parameters of the model. Considering that the sparse degree of convolutional sparse feature maps is closely related to noise, the model introduces learnable weight coefficients into the penalty items of processing LDCT high-frequency maps. The experimental results show that the interpretable DD-DCSR network can well restore the texture details of the image when removing noise/artifacts.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"899-914"},"PeriodicalIF":5.4,"publicationDate":"2024-03-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141197655","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"MAPANet: A Multi-Scale Attention-Guided Progressive Aggregation Network for Multi-Contrast MRI Super-Resolution","authors":"Licheng Liu;Tao Liu;Wei Zhou;Yaonan Wang;Min Liu","doi":"10.1109/TCI.2024.3393723","DOIUrl":"10.1109/TCI.2024.3393723","url":null,"abstract":"Multi-contrast magnetic resonance imaging (MRI) super-resolution (SR), which utilizes complementary information from different contrast images to reconstruct the target images, can provide rich information for quantitative image analysis and accurate medical diagnosis. However, the current mainstream methods are failed in exploiting multi-scale features or global information for data representation, leading to poor outcomes. To address these limitations, we propose a multi-scale attention-guided progressive aggregation network (MAPANet) to progressively restore the target contrast MR images from the corresponding low resolution (LR) observations with the assistance of auxiliary contrast images. Specifically, the proposed MAPANet is composed of several stacked dual-branch aggregation (DBA) blocks, each of which consists of two parallel modules: the multi-scale attention module (MSAM) and the reference feature extraction module (RFEM). The former aims to utilize multi-scale and appropriate non-local information to facilitate the SR reconstruction, while the latter is designed to extract the complementary information from auxiliary contrast images to assist in restoring edge structures and details for target contrast images. Extensive experiments on the public datasets demonstrate that the proposed MAPANet outperforms several state-of-the-art multi-contrast SR methods.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"928-940"},"PeriodicalIF":5.4,"publicationDate":"2024-03-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141197656","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Hybrid Pixel-Wise Registration Learning for Robust Fusion-Based Hyperspectral Image Super-Resolution","authors":"Jiangtao Nie;Wei Wei;Lei Zhang;Chen Ding;Yanning Zhang","doi":"10.1109/TCI.2024.3408095","DOIUrl":"10.1109/TCI.2024.3408095","url":null,"abstract":"Hyperspectral image (HSI) super-resolution (SR) aims to generate a high resolution (HR) HSI in both spectral and spatial domains, in which the fusion-based SR methods have shown great potential in producing a pleasing HR HSI by taking both advantages of the observed low-resolution (LR) HSI and HR multispectral image (MSI). Most existing fusion-based methods implicitly assume that the observed LR HSI and HR MSI are exactly registered, which is, however, difficult to comply with in practice and thus impedes their generalization performance in real applications. To mitigate this problem, we propose a hybrid pixel-wise registration learning framework for fusion-based HSI SR, which shows two aspects of advantages. On the one hand, a pixel-wise registration module (PRM) is developed to directly estimate the transformed coordinate of each pixel, which enables coping with various complex (e.g., rigid or nonrigid) misalignment between two observed images and is pluggable to any other existing architectures. On the other hand, a hybrid learning scheme is conducted to jointly learn both the PRM and the deep image prior-based SR network. Through compositing supervised and unsupervised learning in a two-stage manner, the proposed method is able to exploit both the image-agnostic and image-specific characteristics to robustly cope with unknown misalignment and thus improve the generalization capacity. Experimental results on four benchmark datasets show the superior performance of the proposed method in handling fusion-based HSI SR with various unknown misalignments.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"915-927"},"PeriodicalIF":5.4,"publicationDate":"2024-03-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141197674","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}