IEEE Transactions on Computational Imaging最新文献_第10页

Learning Task-Specific Strategies for Accelerated MRI 学习针对特定任务的加速核磁共振成像策略

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-07-01 DOI: 10.1109/TCI.2024.3410521

Zihui Wu;Tianwei Yin;Yu Sun;Robert Frost;Andre van der Kouwe;Adrian V. Dalca;Katherine L. Bouman

{"title":"Learning Task-Specific Strategies for Accelerated MRI","authors":"Zihui Wu;Tianwei Yin;Yu Sun;Robert Frost;Andre van der Kouwe;Adrian V. Dalca;Katherine L. Bouman","doi":"10.1109/TCI.2024.3410521","DOIUrl":"10.1109/TCI.2024.3410521","url":null,"abstract":"Compressed sensing magnetic resonance imaging (CS-MRI) seeks to recover visual information from subsampled measurements for diagnostic tasks. Traditional CS-MRI methods often separately address measurement subsampling, image reconstruction, and task prediction, resulting in a suboptimal end-to-end performance. In this work, we propose \u0000<sc>Tackle\u0000 as a unified co-design framework for jointly optimizing subsampling, reconstruction, and prediction strategies for the performance on downstream tasks. The naïve approach of simply appending a task prediction module and training with a task-specific loss leads to suboptimal downstream performance. Instead, we develop a training procedure where a backbone architecture is first trained for a generic pre-training task (image reconstruction in our case), and then fine-tuned for different downstream tasks with a prediction head. Experimental results on multiple public MRI datasets show that \u0000<sc>Tackle\u0000 achieves an improved performance on various tasks over traditional CS-MRI methods. We also demonstrate that \u0000<sc>Tackle\u0000 is robust to distribution shifts by showing that it generalizes to a new dataset we experimentally collected using different acquisition setups from the training data. Without additional fine-tuning, \u0000<sc>Tackle\u0000 leads to both numerical and visual improvements compared to existing baselines. We have further implemented a learned 4×-accelerated sequence on a Siemens 3T MRI Skyra scanner. Compared to the fully-sampling scan that takes 335 seconds, our optimized sequence only takes 84 seconds, achieving a four-fold time reduction as desired, while maintaining high performance.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1040-1054"},"PeriodicalIF":4.2,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141531297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Restoration on High Turbidity Water Images Under Near-Field Illumination Using a Light-Field Camera 使用光场相机修复近场照明下的高浊度水图像

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-07-01 DOI: 10.1109/TCI.2024.3420881

Shijun Zhou;Zhen Zhang;Yajing Liu;Jiandong Tian

{"title":"Restoration on High Turbidity Water Images Under Near-Field Illumination Using a Light-Field Camera","authors":"Shijun Zhou;Zhen Zhang;Yajing Liu;Jiandong Tian","doi":"10.1109/TCI.2024.3420881","DOIUrl":"10.1109/TCI.2024.3420881","url":null,"abstract":"Restoring underwater degraded images necessitates accurate estimation of backscatter. Prior research commonly treats backscatter as a constant value across channels. However, addressing backscatter removal becomes intricate when images are captured under conditions of near-field illumination and within densely scattered mediums. In these scenarios, the approximation of backscatter by constant values falls short of efficacy. This paper presents an innovative methodology for characterizing backscatter distribution using curved surfaces while taking into account the scattering conditions at the pixel level. Unlike the previous methods that employ the atmosphere scattering model, we introduce an adaptative function to describe backscatter distribution. By capitalizing on the capabilities of light field cameras in recording light directions, we devise a solution to the focus problem encountered in turbid water environments. Through shear and refocus operations, we not only achieve denoising but also elevate overall image quality. The experimental results clearly demonstrate that our method outperforms state-of-the-art approaches in terms of both visual quality and quantitative metrics.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"984-999"},"PeriodicalIF":4.2,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141503924","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

RIRO: From Retinex-Inspired Reconstruction Optimization Model to Deep Low-Light Image Enhancement Unfolding Network RIRO：从视网膜启发重建优化模型到深度弱光图像增强展开网络

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-07-01 DOI: 10.1109/TCI.2024.3420942

Lijun Zhao;Bintao Chen;Jinjing Zhang;Anhong Wang;Huihui Bai

{"title":"RIRO: From Retinex-Inspired Reconstruction Optimization Model to Deep Low-Light Image Enhancement Unfolding Network","authors":"Lijun Zhao;Bintao Chen;Jinjing Zhang;Anhong Wang;Huihui Bai","doi":"10.1109/TCI.2024.3420942","DOIUrl":"10.1109/TCI.2024.3420942","url":null,"abstract":"Low contrast, noise pollution and color distortion of low-light images tremendously affect human visual perception. The Retinex and its variant models are widely used for low-light image enhancement (LLIE). However, the performances of traditional Retinex algorithms are limited by intrinsic non-learnable characteristic. Recently, the latest LLIE methods directly unfold Retinex model as the popular networks such as URetinex-Net and RAUNA to resolve the black-box problem of conventional neural networks. Different from these methods focusing on the unfolding of image decomposition, we treat the classic LLIE as an image reconstruction task. Built upon Retinex theory, we propose a Retinex-Inspired Reconstruction Optimization (RIRO) model, which is unrolled as the RIRO network. This network consists of Low-light Decomposition and Enhancement Sub-Network (LDE Sub-Net) and Image Reconstruction Unrolling Sub-Network (IRU Sub-Net). The LDE Sub-Net is leveraged for the input initialization of the IRU Sub-Net. In RIRO model, we introduce a Dual-Domain Proximal (DDP) block to replace classic proximal operator, in which Fourier transform is utilized to transform spatial domain information into frequency domain information so as to simultaneously extract dual features on both spatial and frequency domains. Besides, we design a residual-aware weighted dual-fusion module and an adaptive weighted triple-fusion module to fuse different kinds of features. Numerous experiments on benchmark datasets have shown that the proposed method outperforms many advanced LIE methods.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"969-983"},"PeriodicalIF":4.2,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141517954","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Application of Kirchhoff Migration Principle for Hardware-Efficient Near-Field Radar Imaging 基尔霍夫迁移原理在硬件高效近场雷达成像中的应用

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-06-26 DOI: 10.1109/TCI.2024.3419580

Amir Masoud Molaei;María García-Fernández;Guillermo Álvarez-Narciandi;Rupesh Kumar;Vasiliki Skouroliakou;Vincent Fusco;Muhammad Ali Babar Abbasi;Okan Yurduseven

{"title":"Application of Kirchhoff Migration Principle for Hardware-Efficient Near-Field Radar Imaging","authors":"Amir Masoud Molaei;María García-Fernández;Guillermo Álvarez-Narciandi;Rupesh Kumar;Vasiliki Skouroliakou;Vincent Fusco;Muhammad Ali Babar Abbasi;Okan Yurduseven","doi":"10.1109/TCI.2024.3419580","DOIUrl":"10.1109/TCI.2024.3419580","url":null,"abstract":"Achieving high imaging resolution in conventional monostatic radar imaging with mechanical scanning requires excessive acquisition time. Although real aperture radar systems might not suffer from such a limitation in acquisition time, they may still face challenges in achieving high imaging resolution, especially in near-field (NF) scenarios, due to diffraction-limited performance. Even with sophisticated electronic scanning techniques, increasing the aperture size to improve resolution can lead to complex hardware setups and may not always be feasible in certain practical scenarios. Multistatic systems can virtually increase the effective aperture but introduce challenges due to the required number of antennas and channels, making them expensive, bulky and power-intensive. An alternative solution that has been proposed in recent years is the compression of the physical layer using metasurface transducers. This paper presents a novel NF radar imaging approach leveraging dynamic metasurface antennas with multiple tuning states called \u0000<italic>masks\u0000, in a bistatic structure, using the Kirchhoff migration principle. The method involves expanding the compressed measured signal from the mask-frequency domain to the spatial-frequency domain to decode the scene's spatial content. The Kirchhoff integral is then developed based on the introduced special imaging structure to retrieve the three-dimensional spatial information of the target. Comprehensive numerical simulations analyze the masks' characteristics and their behavior under different conditions. The performance of the image reconstruction algorithm is evaluated for visual quality and computing time using both central processing units and graphics processing units. The results of computer simulations confirm the high reliability of the proposed approach in various cases.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1000-1015"},"PeriodicalIF":4.2,"publicationDate":"2024-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141503925","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Sequential Experimental Design for X-Ray CT Using Deep Reinforcement Learning 利用深度强化学习进行 X 射线 CT 序列实验设计

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-06-26 DOI: 10.1109/TCI.2024.3414273

Tianyuan Wang;Felix Lucka;Tristan van Leeuwen

{"title":"Sequential Experimental Design for X-Ray CT Using Deep Reinforcement Learning","authors":"Tianyuan Wang;Felix Lucka;Tristan van Leeuwen","doi":"10.1109/TCI.2024.3414273","DOIUrl":"https://doi.org/10.1109/TCI.2024.3414273","url":null,"abstract":"In X-ray Computed Tomography (CT), projections from many angles are acquired and used for 3D reconstruction. To make CT suitable for in-line quality control, reducing the number of angles while maintaining reconstruction quality is necessary. Sparse-angle tomography is a popular approach for obtaining 3D reconstructions from limited data. To optimize its performance, one can adapt scan angles sequentially to select the most informative angles for each scanned object. Mathematically, this corresponds to solving an optimal experimental design (OED) problem. OED problems are high-dimensional, non-convex, bi-level optimization problems that cannot be solved online, i.e., during the scan. To address these challenges, we pose the OED problem as a partially observable Markov decision process in a Bayesian framework, and solve it through deep reinforcement learning. The approach learns efficient non-greedy policies to solve a given class of OED problems through extensive offline training rather than solving a given OED problem directly via numerical optimization. As such, the trained policy can successfully find the most informative scan angles online. We use a policy training method based on the Actor-Critic approach and evaluate its performance on 2D tomography with synthetic data.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"953-968"},"PeriodicalIF":4.2,"publicationDate":"2024-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141453459","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Single-Shot Tomography of Discrete Dynamic Objects 离散动态物体的单镜头断层扫描

IF 4.2 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-06-13 DOI: 10.1109/TCI.2024.3414320

Ajinkya Kadu;Felix Lucka;Kees Joost Batenburg

{"title":"Single-Shot Tomography of Discrete Dynamic Objects","authors":"Ajinkya Kadu;Felix Lucka;Kees Joost Batenburg","doi":"10.1109/TCI.2024.3414320","DOIUrl":"https://doi.org/10.1109/TCI.2024.3414320","url":null,"abstract":"This paper presents a novel method for the reconstruction of high-resolution temporal images in dynamic tomographic imaging, particularly for discrete objects with smooth boundaries that vary over time. Addressing the challenge of limited measurements per time point, we propose a technique that incorporates spatial and temporal information of the dynamic objects. Our method uses the explicit assumption of homogeneous attenuation values of discrete objects. We achieve this computationally through the application of the level-set method for image segmentation and the representation of motion via a sinusoidal basis. The result is a computationally efficient and easily optimizable variational framework that enables the reconstruction of high-quality 2D or 3D image sequences with a single projection per frame. Compared to variational regularization-based methods using similar image models, our approach demonstrates superior performance on both synthetic and pseudo-dynamic real X-ray tomography datasets. The implications of this research extend to improved visualization and analysis of dynamic processes in tomographic imaging, finding potential applications in diverse scientific and industrial domains. The supporting data and code are provided.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"941-952"},"PeriodicalIF":4.2,"publicationDate":"2024-06-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141453458","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

SeNAS-Net: Self-Supervised Noise and Artifact Suppression Network for Material Decomposition in Spectral CT SeNAS-Net：用于光谱 CT 材料分解的自监督噪声和伪影抑制网络

IF 5.4 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-04-29 DOI: 10.1109/TCI.2024.3394772

Xu Ji;Yuchen Lu;Yikun Zhang;Xu Zhuo;Shengqi Kan;Weilong Mao;Gouenou Coatrieux;Jean-Louis Coatrieux;Guotao Quan;Yan Xi;Shuo Li;Tianling Lyu;Yang Chen

{"title":"SeNAS-Net: Self-Supervised Noise and Artifact Suppression Network for Material Decomposition in Spectral CT","authors":"Xu Ji;Yuchen Lu;Yikun Zhang;Xu Zhuo;Shengqi Kan;Weilong Mao;Gouenou Coatrieux;Jean-Louis Coatrieux;Guotao Quan;Yan Xi;Shuo Li;Tianling Lyu;Yang Chen","doi":"10.1109/TCI.2024.3394772","DOIUrl":"10.1109/TCI.2024.3394772","url":null,"abstract":"For material decomposition in spectral computed tomography, the x-ray attenuation coefficient of an unknown material can be decomposed as a combination of a group of basis materials, in order to analyze its material properties. Material decomposition generally leads to amplification of image noise and artifacts. Meanwhile, it is often difficult to acquire the ground truth values of the material basis images, preventing the application of supervised learning-based noise reduction methods. To resolve such problem, we proposed a self-supervised noise and artifact suppression network for spectral computed tomography. The proposed method consists of a projection-domain self-supervised denoising network along with physics-driven constraints to mitigate the secondary artifacts, including a noise modulation item to incorporate the anisotropic noise amplitudes in the projection domain, a sinogram mask image to suppress streaky artifacts and a data fidelity loss item to further mitigate noise and to improve signal accuracy. The performance of the proposed method was evaluated based on both numerical experiment tests and laboratory experiment tests. Results demonstrated that the proposed method has promising performance in noise and artifact suppression for material decomposition in spectral computed tomography. Comprehensive ablation studies were performed to demonstrate the function of each physical constraint.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"677-689"},"PeriodicalIF":5.4,"publicationDate":"2024-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140831029","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Local Monotone Operator Learning Using Non-Monotone Operators: MnM-MOL 利用非单调算子进行局部单调算子学习MnM-MOL

IF 5.4 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-04-29 DOI: 10.1109/TCI.2024.3393742

Maneesh John;Jyothi Rikhab Chand;Mathews Jacob

{"title":"Local Monotone Operator Learning Using Non-Monotone Operators: MnM-MOL","authors":"Maneesh John;Jyothi Rikhab Chand;Mathews Jacob","doi":"10.1109/TCI.2024.3393742","DOIUrl":"10.1109/TCI.2024.3393742","url":null,"abstract":"The recovery of magnetic resonance (MR) images from undersampled measurements is a key problem that has been the subject of extensive research in recent years. Unrolled approaches, which rely on end-to-end training of convolutional neural network (CNN) blocks within iterative reconstruction algorithms, offer state-of-the-art performance. These algorithms require a large amount of memory during training, making them difficult to employ in high-dimensional applications. Deep equilibrium (DEQ) models and the recent monotone operator learning (MOL) approach were introduced to eliminate the need for unrolling, thus reducing the memory demand during training. Both approaches require a Lipschitz constraint on the network to ensure that the forward and backpropagation iterations converge. Unfortunately, the constraint often results in reduced performance compared to the unrolled methods. The main focus of this work is to relax the constraint on the CNN block in two different ways. Inspired by convex-non-convex regularization strategies, we now impose the monotone constraint on the sum of the gradient of the data term and the CNN block, rather than constrain the CNN itself to be a monotone operator. This approach enables the CNN to learn possibly non-monotone score functions, which can translate to improved performance. In addition, we only restrict the operator to be monotone in a local neighborhood around the image manifold. Our theoretical results show that the proposed algorithm is guaranteed to converge to the fixed point and that the solution is robust to input perturbations, provided that it is initialized close to the true solution. Our empirical results show that the relaxed constraints translate to improved performance and that the approach enjoys robustness to input perturbations similar to MOL.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"742-751"},"PeriodicalIF":5.4,"publicationDate":"2024-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140830853","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Towards Robustness and Efficiency of Coherence-Guided Complex Convolutional Sparse Coding for Interferometric Phase Restoration 实现相干引导的复杂卷积稀疏编码在干涉相位恢复中的鲁棒性和效率

IF 5.4 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-04-29 DOI: 10.1109/TCI.2024.3393760

Xiang Ding;Jian Kang;Yusong Bai;Anping Zhang;Jialin Liu;Naoto Yokoya

{"title":"Towards Robustness and Efficiency of Coherence-Guided Complex Convolutional Sparse Coding for Interferometric Phase Restoration","authors":"Xiang Ding;Jian Kang;Yusong Bai;Anping Zhang;Jialin Liu;Naoto Yokoya","doi":"10.1109/TCI.2024.3393760","DOIUrl":"10.1109/TCI.2024.3393760","url":null,"abstract":"Recently, complex convolutional sparse coding (ComCSC) has demonstrated its effectiveness in interferometric phase restoration, owing to its prominent performance in noise mitigation and detailed phase preservation. By incorporating the estimated coherence into ComCSC as prior knowledge for re-weighting individual complex residues, coherence-guided complex convolutional sparse coding (CoComCSC) further improves the quality of restored phases, especially over heterogeneous land-covers with rapidly varying coherence. However, due to the exploited \u0000<inline-formula><tex-math>$L_{2}$</tex-math></inline-formula>\u0000 norm of the data fidelity term, the original CoComCSC is not robust to outliers when relatively low coherence values are sparsely distributed over high ones. We propose CoComCSC-L1 and CoComCSC-Huber to improve the robustness of CoComCSC based on the \u0000<inline-formula><tex-math>$L_{1}$</tex-math></inline-formula>\u0000 and Huber norms. Moreover, we propose an efficient solver to decrease the computational cost of solving the linear system subproblem within ComCSC-based optimization problems. By comparing the proposed methods to other state-of-the-art methods using both simulated and real data, the proposed methods demonstrate their effectiveness. Additionally, the proposed solver has the potential to improve optimization speed by approximately 10% compared to the state-of-the-art solver.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"690-699"},"PeriodicalIF":5.4,"publicationDate":"2024-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140830915","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Disparity-Guided Multi-View Interaction Network for Light Field Reflection Removal 用于消除光场反射的差异引导多视角交互网络

IF 5.4 2区计算机科学

IEEE Transactions on Computational Imaging Pub Date : 2024-04-29 DOI: 10.1109/TCI.2024.3394773

Yutong Liu;Wenming Weng;Ruisheng Gao;Zeyu Xiao;Yueyi Zhang;Zhiwei Xiong

{"title":"Disparity-Guided Multi-View Interaction Network for Light Field Reflection Removal","authors":"Yutong Liu;Wenming Weng;Ruisheng Gao;Zeyu Xiao;Yueyi Zhang;Zhiwei Xiong","doi":"10.1109/TCI.2024.3394773","DOIUrl":"10.1109/TCI.2024.3394773","url":null,"abstract":"Light field (LF) imaging presents a promising avenue for reflection removal, owing to its ability of reliable depth perception and utilization of complementary texture details from multiple sub-aperture images (SAIs). However, the domain shifts between real-world and synthetic scenes, as well as the challenge of embedding transmission information across SAIs pose the main obstacles in this task. In this paper, we conquer the above challenges from the perspectives of data and network, respectively. To mitigate domain shifts, we propose an efficient data synthesis strategy for simulating realistic reflection scenes, and build the largest ever LF reflection dataset containing 420 synthetic scenes and 70 real-world scenes. To enable the transmission information embedding across SAIs, we propose a novel \u0000<underline>D\u0000isparity-guided \u0000<underline>M\u0000ulti-view \u0000<underline>I\u0000nteraction \u0000<underline>Net\u0000work (DMINet) for LF reflection removal. DMINet mainly consists of a transmission disparity estimation (TDE) module and a center-side interaction (CSI) module. The TDE module aims to predict transmission disparity by filtering out reflection disturbances, while the CSI module is responsible for the transmission integration which adopts the central view as the bridge for the propagation conducted between different SAIs. Compared with existing reflection removal methods for LF input, DMINet achieves a distinct performance boost with merits of efficiency and robustness, especially for scenes with complex depth variations.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"726-741"},"PeriodicalIF":5.4,"publicationDate":"2024-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140841932","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0