Jingchao Hou;Garas Gendy;Guo Chen;Liangchao Wang;Guanghui He
{"title":"DTDeMo: A Deep Learning-Based Two-Stage Image Demosaicing Model With Interpolation and Enhancement","authors":"Jingchao Hou;Garas Gendy;Guo Chen;Liangchao Wang;Guanghui He","doi":"10.1109/TCI.2024.3426360","DOIUrl":"10.1109/TCI.2024.3426360","url":null,"abstract":"Image demosaicing is one of the most ubiquitous and performance-critical image processing tasks. However, traditional demosaicing methods use fixed weights to finish the interpolation, while deep learning demosaicing restoration always breaks the image array arrangement rule, and they can't fully use the existing pixel information. To rectify these weaknesses, in this paper, we propose the convolution interpolation block (CIB) to obey the RAW data arrangement rule and the deep demosaicing residual block (DDRB) to repeatedly utilize existing pixel information for demosaicing. Based on the CIB and DDRB, we present a novel two-stage demosaicing model (DTDeMo), including differential interpolation and enhancement processes. Specifically, the interpolation process contains several CIBs and DDRBs with trainable interpolation parameters. Meanwhile, the enhancement process consists of a transformer-based block and a series of DDRBs to enhance the interpolation results. The effectiveness of CIBs, DDRBs, the proposed interpolation process, and the enhancement process is confirmed through an ablation study. A thorough comparison with several methods shows that our DTDeMo outperforms state of the art quantitatively and qualitatively.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1026-1039"},"PeriodicalIF":4.2,"publicationDate":"2024-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141588024","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mohammad Khateri;Morteza Ghahremani;Alejandra Sierra;Jussi Tohka
{"title":"No-Clean-Reference Image Super-Resolution: Application to Electron Microscopy","authors":"Mohammad Khateri;Morteza Ghahremani;Alejandra Sierra;Jussi Tohka","doi":"10.1109/TCI.2024.3426349","DOIUrl":"10.1109/TCI.2024.3426349","url":null,"abstract":"The inability to acquire clean high-resolution (HR) electron microscopy (EM) images over a large brain tissue volume hampers many neuroscience studies. To address this challenge, we propose a deep-learning-based image super-resolution (SR) approach to computationally reconstruct a clean HR 3D-EM image with a large field of view (FoV) from noisy low-resolution (LR) acquisition. Our contributions are I) investigation of training with no-clean references; II) introduction of a novel network architecture, named EMSR, for enhancing the resolution of LR EM images while reducing inherent noise. The EMSR leverages distinctive features in brain EM images–repetitive textural and geometrical patterns amidst less informative backgrounds– via multiscale edge-attention and self-attention mechanisms to emphasize edge features over the background; and, III) comparison of different training strategies including using acquired LR and HR image pairs, i.e., real pairs with no-clean references contaminated with real corruptions, pairs of synthetic LR and acquired HR, as well as acquired LR and denoised HR pairs. Experiments with nine brain datasets showed that training with real pairs can produce high-quality super-resolved results, demonstrating the feasibility of training with nonclean references. Additionally, comparable results were observed, both visually and numerically, when employing denoised and noisy references for training. Moreover, utilizing the network trained with synthetically generated LR images from HR counterparts proved effective in yielding satisfactory SR results, even in certain cases, outperforming training with real pairs. The proposed SR network was compared quantitatively and qualitatively with several established SR techniques, demonstrating either the superiority or competitiveness of the proposed method in recovering fine details while mitigating noise.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1094-1110"},"PeriodicalIF":4.2,"publicationDate":"2024-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10592622","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141588023","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Deep Learned Non-Linear Propagation Model Regularizer for Compressive Spectral Imaging","authors":"Romario Gualdrón-Hurtado;Henry Arguello;Jorge Bacca","doi":"10.1109/TCI.2024.3422900","DOIUrl":"10.1109/TCI.2024.3422900","url":null,"abstract":"Coded aperture snapshot spectral imager (CASSI), efficiently captures 3D spectral images by sensing 2D projections of the scene. While CASSI offers a substantial reduction in acquisition time, compared to traditional scanning optical systems, it requires a reconstruction post-processing step. Furthermore, to obtain high-quality reconstructions, an accurate propagation model is required. Notably, CASSI exhibits a variant spatio-spectral sensor response, making it difficult to acquire an accurate propagation model. To address these inherent limitations, this work proposes to learn a deep non-linear fully differentiable propagation model that can be used as a regularizer within an optimization-based reconstruction algorithm. The proposed approach trains the non-linear spatially-variant propagation model using paired compressed measurements and spectral images, by employing side information only in the calibration step. From the deep propagation model incorporation into a plug-and-play alternating direction method of multipliers framework, our proposed method outperforms traditional CASSI linear-based models. Extensive simulations and a testbed implementation validate the efficacy of the proposed methodology.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1016-1025"},"PeriodicalIF":4.2,"publicationDate":"2024-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141573765","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"PGIUN: Physics-Guided Implicit Unrolling Network for Accelerated MRI","authors":"Jiawei Jiang;Zihan He;Yueqian Quan;Jie Wu;Jianwei Zheng","doi":"10.1109/TCI.2024.3422840","DOIUrl":"10.1109/TCI.2024.3422840","url":null,"abstract":"To cope with the challenges stemming from prolonged acquisition periods, compressed sensing MRI has emerged as a popular technique to accelerate the reconstruction of high-quality images from under-sampled k-space data. Most current solutions endeavor to solve this issue with the pursuit of certain prior properties, yet the treatments are all enforced in the original space, resulting in limited feature information. To boost the performance yet with the guarantee of high running efficiency, in this study, we propose a Physics-Guided Implicit Unrolling Network (PGIUN). Specifically, by an elaborately designed reversible network, the inputs are first mapped to a channel-lifted implicit space, which taps the potential of capturing spatial-invariant features sufficiently. Within this implicit space, we then unfold an accelerated optimization algorithm to iterate an efficient and feasible solution, in which a parallelly dual-domain update is equipped for better feature fusion. Finally, an inverse embedding transformation of the recovered high-dimensional representation is employed to achieve the desired estimation. PGIUN enjoys high interpretability benefiting from the physically induced modules, which not only facilitates an intuitive understanding of the internal working mechanism but also endows it with high generalization ability. Extensive experiments conducted across diverse datasets and varying sampling rates/patterns consistently establish the superiority of our approach over state-of-the-art methods in both visual and quantitative evaluations.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1055-1068"},"PeriodicalIF":4.2,"publicationDate":"2024-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141549933","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zihui Wu;Tianwei Yin;Yu Sun;Robert Frost;Andre van der Kouwe;Adrian V. Dalca;Katherine L. Bouman
{"title":"Learning Task-Specific Strategies for Accelerated MRI","authors":"Zihui Wu;Tianwei Yin;Yu Sun;Robert Frost;Andre van der Kouwe;Adrian V. Dalca;Katherine L. Bouman","doi":"10.1109/TCI.2024.3410521","DOIUrl":"10.1109/TCI.2024.3410521","url":null,"abstract":"Compressed sensing magnetic resonance imaging (CS-MRI) seeks to recover visual information from subsampled measurements for diagnostic tasks. Traditional CS-MRI methods often separately address measurement subsampling, image reconstruction, and task prediction, resulting in a suboptimal end-to-end performance. In this work, we propose \u0000<sc>Tackle</small>\u0000 as a unified co-design framework for jointly optimizing subsampling, reconstruction, and prediction strategies for the performance on downstream tasks. The naïve approach of simply appending a task prediction module and training with a task-specific loss leads to suboptimal downstream performance. Instead, we develop a training procedure where a backbone architecture is first trained for a generic pre-training task (image reconstruction in our case), and then fine-tuned for different downstream tasks with a prediction head. Experimental results on multiple public MRI datasets show that \u0000<sc>Tackle</small>\u0000 achieves an improved performance on various tasks over traditional CS-MRI methods. We also demonstrate that \u0000<sc>Tackle</small>\u0000 is robust to distribution shifts by showing that it generalizes to a new dataset we experimentally collected using different acquisition setups from the training data. Without additional fine-tuning, \u0000<sc>Tackle</small>\u0000 leads to both numerical and visual improvements compared to existing baselines. We have further implemented a learned 4×-accelerated sequence on a Siemens 3T MRI Skyra scanner. Compared to the fully-sampling scan that takes 335 seconds, our optimized sequence only takes 84 seconds, achieving a four-fold time reduction as desired, while maintaining high performance.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1040-1054"},"PeriodicalIF":4.2,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141531297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Restoration on High Turbidity Water Images Under Near-Field Illumination Using a Light-Field Camera","authors":"Shijun Zhou;Zhen Zhang;Yajing Liu;Jiandong Tian","doi":"10.1109/TCI.2024.3420881","DOIUrl":"10.1109/TCI.2024.3420881","url":null,"abstract":"Restoring underwater degraded images necessitates accurate estimation of backscatter. Prior research commonly treats backscatter as a constant value across channels. However, addressing backscatter removal becomes intricate when images are captured under conditions of near-field illumination and within densely scattered mediums. In these scenarios, the approximation of backscatter by constant values falls short of efficacy. This paper presents an innovative methodology for characterizing backscatter distribution using curved surfaces while taking into account the scattering conditions at the pixel level. Unlike the previous methods that employ the atmosphere scattering model, we introduce an adaptative function to describe backscatter distribution. By capitalizing on the capabilities of light field cameras in recording light directions, we devise a solution to the focus problem encountered in turbid water environments. Through shear and refocus operations, we not only achieve denoising but also elevate overall image quality. The experimental results clearly demonstrate that our method outperforms state-of-the-art approaches in terms of both visual quality and quantitative metrics.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"984-999"},"PeriodicalIF":4.2,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141503924","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lijun Zhao;Bintao Chen;Jinjing Zhang;Anhong Wang;Huihui Bai
{"title":"RIRO: From Retinex-Inspired Reconstruction Optimization Model to Deep Low-Light Image Enhancement Unfolding Network","authors":"Lijun Zhao;Bintao Chen;Jinjing Zhang;Anhong Wang;Huihui Bai","doi":"10.1109/TCI.2024.3420942","DOIUrl":"10.1109/TCI.2024.3420942","url":null,"abstract":"Low contrast, noise pollution and color distortion of low-light images tremendously affect human visual perception. The Retinex and its variant models are widely used for low-light image enhancement (LLIE). However, the performances of traditional Retinex algorithms are limited by intrinsic non-learnable characteristic. Recently, the latest LLIE methods directly unfold Retinex model as the popular networks such as URetinex-Net and RAUNA to resolve the black-box problem of conventional neural networks. Different from these methods focusing on the unfolding of image decomposition, we treat the classic LLIE as an image reconstruction task. Built upon Retinex theory, we propose a Retinex-Inspired Reconstruction Optimization (RIRO) model, which is unrolled as the RIRO network. This network consists of Low-light Decomposition and Enhancement Sub-Network (LDE Sub-Net) and Image Reconstruction Unrolling Sub-Network (IRU Sub-Net). The LDE Sub-Net is leveraged for the input initialization of the IRU Sub-Net. In RIRO model, we introduce a Dual-Domain Proximal (DDP) block to replace classic proximal operator, in which Fourier transform is utilized to transform spatial domain information into frequency domain information so as to simultaneously extract dual features on both spatial and frequency domains. Besides, we design a residual-aware weighted dual-fusion module and an adaptive weighted triple-fusion module to fuse different kinds of features. Numerous experiments on benchmark datasets have shown that the proposed method outperforms many advanced LIE methods.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"969-983"},"PeriodicalIF":4.2,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141517954","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Amir Masoud Molaei;María García-Fernández;Guillermo Álvarez-Narciandi;Rupesh Kumar;Vasiliki Skouroliakou;Vincent Fusco;Muhammad Ali Babar Abbasi;Okan Yurduseven
{"title":"Application of Kirchhoff Migration Principle for Hardware-Efficient Near-Field Radar Imaging","authors":"Amir Masoud Molaei;María García-Fernández;Guillermo Álvarez-Narciandi;Rupesh Kumar;Vasiliki Skouroliakou;Vincent Fusco;Muhammad Ali Babar Abbasi;Okan Yurduseven","doi":"10.1109/TCI.2024.3419580","DOIUrl":"10.1109/TCI.2024.3419580","url":null,"abstract":"Achieving high imaging resolution in conventional monostatic radar imaging with mechanical scanning requires excessive acquisition time. Although real aperture radar systems might not suffer from such a limitation in acquisition time, they may still face challenges in achieving high imaging resolution, especially in near-field (NF) scenarios, due to diffraction-limited performance. Even with sophisticated electronic scanning techniques, increasing the aperture size to improve resolution can lead to complex hardware setups and may not always be feasible in certain practical scenarios. Multistatic systems can virtually increase the effective aperture but introduce challenges due to the required number of antennas and channels, making them expensive, bulky and power-intensive. An alternative solution that has been proposed in recent years is the compression of the physical layer using metasurface transducers. This paper presents a novel NF radar imaging approach leveraging dynamic metasurface antennas with multiple tuning states called \u0000<italic>masks</i>\u0000, in a bistatic structure, using the Kirchhoff migration principle. The method involves expanding the compressed measured signal from the mask-frequency domain to the spatial-frequency domain to decode the scene's spatial content. The Kirchhoff integral is then developed based on the introduced special imaging structure to retrieve the three-dimensional spatial information of the target. Comprehensive numerical simulations analyze the masks' characteristics and their behavior under different conditions. The performance of the image reconstruction algorithm is evaluated for visual quality and computing time using both central processing units and graphics processing units. The results of computer simulations confirm the high reliability of the proposed approach in various cases.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1000-1015"},"PeriodicalIF":4.2,"publicationDate":"2024-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141503925","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Sequential Experimental Design for X-Ray CT Using Deep Reinforcement Learning","authors":"Tianyuan Wang;Felix Lucka;Tristan van Leeuwen","doi":"10.1109/TCI.2024.3414273","DOIUrl":"https://doi.org/10.1109/TCI.2024.3414273","url":null,"abstract":"In X-ray Computed Tomography (CT), projections from many angles are acquired and used for 3D reconstruction. To make CT suitable for in-line quality control, reducing the number of angles while maintaining reconstruction quality is necessary. Sparse-angle tomography is a popular approach for obtaining 3D reconstructions from limited data. To optimize its performance, one can adapt scan angles sequentially to select the most informative angles for each scanned object. Mathematically, this corresponds to solving an optimal experimental design (OED) problem. OED problems are high-dimensional, non-convex, bi-level optimization problems that cannot be solved online, i.e., during the scan. To address these challenges, we pose the OED problem as a partially observable Markov decision process in a Bayesian framework, and solve it through deep reinforcement learning. The approach learns efficient non-greedy policies to solve a given class of OED problems through extensive offline training rather than solving a given OED problem directly via numerical optimization. As such, the trained policy can successfully find the most informative scan angles online. We use a policy training method based on the Actor-Critic approach and evaluate its performance on 2D tomography with synthetic data.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"953-968"},"PeriodicalIF":4.2,"publicationDate":"2024-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141453459","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Single-Shot Tomography of Discrete Dynamic Objects","authors":"Ajinkya Kadu;Felix Lucka;Kees Joost Batenburg","doi":"10.1109/TCI.2024.3414320","DOIUrl":"https://doi.org/10.1109/TCI.2024.3414320","url":null,"abstract":"This paper presents a novel method for the reconstruction of high-resolution temporal images in dynamic tomographic imaging, particularly for discrete objects with smooth boundaries that vary over time. Addressing the challenge of limited measurements per time point, we propose a technique that incorporates spatial and temporal information of the dynamic objects. Our method uses the explicit assumption of homogeneous attenuation values of discrete objects. We achieve this computationally through the application of the level-set method for image segmentation and the representation of motion via a sinusoidal basis. The result is a computationally efficient and easily optimizable variational framework that enables the reconstruction of high-quality 2D or 3D image sequences with a single projection per frame. Compared to variational regularization-based methods using similar image models, our approach demonstrates superior performance on both synthetic and pseudo-dynamic real X-ray tomography datasets. The implications of this research extend to improved visualization and analysis of dynamic processes in tomographic imaging, finding potential applications in diverse scientific and industrial domains. The supporting data and code are provided.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"941-952"},"PeriodicalIF":4.2,"publicationDate":"2024-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141453458","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}