{"title":"How to best combine demosaicing and denoising?","authors":"Yu Guo, Qiyu Jin, Jean-Michel Morel, Gabriele Facciolo","doi":"10.3934/ipi.2023044","DOIUrl":null,"url":null,"abstract":"Image demosaicing and denoising play a critical role in the raw imaging pipeline. These processes have often been treated as independent, without considering their interactions. Indeed, most classic denoising methods handle noisy RGB images, not raw images. Conversely, most demosaicing methods address the demosaicing of noise free images. The real problem is to jointly denoise and demosaic noisy raw images. But the question of how to proceed is still not clarified. In this paper, we carry out extensive experiments and a mathematical analysis to tackle this problem by low complexity algorithms. Indeed, both problems have only been addressed jointly by end-to-end heavy-weight convolutional neural networks (CNNs), which are currently incompatible with low-power portable imaging devices and remain by nature domain (or device) dependent. Our study leads us to conclude that, with moderate noise, demosaicing should be applied first, followed by denoising. This requires a simple adaptation of classic denoising algorithms to demosaiced noise, which we justify and specify. Although our main conclusion is 'demosaic first, then denoise,' we also discover that for high noise, there is a moderate PSNR gain by a more complex strategy: partial CFA denoising followed by demosaicing and by a second denoising on the RGB image. These surprising results are obtained by a black-box optimization of the pipeline, which could be applied to any other pipeline. We validate our results on simulated and real noisy CFA images obtained from several benchmarks.","PeriodicalId":50274,"journal":{"name":"Inverse Problems and Imaging","volume":"19 1","pages":"0"},"PeriodicalIF":1.2000,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Inverse Problems and Imaging","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.3934/ipi.2023044","RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"MATHEMATICS, APPLIED","Score":null,"Total":0}
引用次数: 0
Abstract
Image demosaicing and denoising play a critical role in the raw imaging pipeline. These processes have often been treated as independent, without considering their interactions. Indeed, most classic denoising methods handle noisy RGB images, not raw images. Conversely, most demosaicing methods address the demosaicing of noise free images. The real problem is to jointly denoise and demosaic noisy raw images. But the question of how to proceed is still not clarified. In this paper, we carry out extensive experiments and a mathematical analysis to tackle this problem by low complexity algorithms. Indeed, both problems have only been addressed jointly by end-to-end heavy-weight convolutional neural networks (CNNs), which are currently incompatible with low-power portable imaging devices and remain by nature domain (or device) dependent. Our study leads us to conclude that, with moderate noise, demosaicing should be applied first, followed by denoising. This requires a simple adaptation of classic denoising algorithms to demosaiced noise, which we justify and specify. Although our main conclusion is 'demosaic first, then denoise,' we also discover that for high noise, there is a moderate PSNR gain by a more complex strategy: partial CFA denoising followed by demosaicing and by a second denoising on the RGB image. These surprising results are obtained by a black-box optimization of the pipeline, which could be applied to any other pipeline. We validate our results on simulated and real noisy CFA images obtained from several benchmarks.
期刊介绍:
Inverse Problems and Imaging publishes research articles of the highest quality that employ innovative mathematical and modeling techniques to study inverse and imaging problems arising in engineering and other sciences. Every published paper has a strong mathematical orientation employing methods from such areas as control theory, discrete mathematics, differential geometry, harmonic analysis, functional analysis, integral geometry, mathematical physics, numerical analysis, optimization, partial differential equations, and stochastic and statistical methods. The field of applications includes medical and other imaging, nondestructive testing, geophysical prospection and remote sensing as well as image analysis and image processing.
This journal is committed to recording important new results in its field and will maintain the highest standards of innovation and quality. To be published in this journal, a paper must be correct, novel, nontrivial and of interest to a substantial number of researchers and readers.