如何最好地将去马赛克和去噪结合起来?

IF 1.2 4区 数学 Q2 MATHEMATICS, APPLIED
Yu Guo, Qiyu Jin, Jean-Michel Morel, Gabriele Facciolo
{"title":"如何最好地将去马赛克和去噪结合起来?","authors":"Yu Guo, Qiyu Jin, Jean-Michel Morel, Gabriele Facciolo","doi":"10.3934/ipi.2023044","DOIUrl":null,"url":null,"abstract":"Image demosaicing and denoising play a critical role in the raw imaging pipeline. These processes have often been treated as independent, without considering their interactions. Indeed, most classic denoising methods handle noisy RGB images, not raw images. Conversely, most demosaicing methods address the demosaicing of noise free images. The real problem is to jointly denoise and demosaic noisy raw images. But the question of how to proceed is still not clarified. In this paper, we carry out extensive experiments and a mathematical analysis to tackle this problem by low complexity algorithms. Indeed, both problems have only been addressed jointly by end-to-end heavy-weight convolutional neural networks (CNNs), which are currently incompatible with low-power portable imaging devices and remain by nature domain (or device) dependent. Our study leads us to conclude that, with moderate noise, demosaicing should be applied first, followed by denoising. This requires a simple adaptation of classic denoising algorithms to demosaiced noise, which we justify and specify. Although our main conclusion is 'demosaic first, then denoise,' we also discover that for high noise, there is a moderate PSNR gain by a more complex strategy: partial CFA denoising followed by demosaicing and by a second denoising on the RGB image. These surprising results are obtained by a black-box optimization of the pipeline, which could be applied to any other pipeline. We validate our results on simulated and real noisy CFA images obtained from several benchmarks.","PeriodicalId":50274,"journal":{"name":"Inverse Problems and Imaging","volume":"19 1","pages":"0"},"PeriodicalIF":1.2000,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"How to best combine demosaicing and denoising?\",\"authors\":\"Yu Guo, Qiyu Jin, Jean-Michel Morel, Gabriele Facciolo\",\"doi\":\"10.3934/ipi.2023044\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Image demosaicing and denoising play a critical role in the raw imaging pipeline. These processes have often been treated as independent, without considering their interactions. Indeed, most classic denoising methods handle noisy RGB images, not raw images. Conversely, most demosaicing methods address the demosaicing of noise free images. The real problem is to jointly denoise and demosaic noisy raw images. But the question of how to proceed is still not clarified. In this paper, we carry out extensive experiments and a mathematical analysis to tackle this problem by low complexity algorithms. Indeed, both problems have only been addressed jointly by end-to-end heavy-weight convolutional neural networks (CNNs), which are currently incompatible with low-power portable imaging devices and remain by nature domain (or device) dependent. Our study leads us to conclude that, with moderate noise, demosaicing should be applied first, followed by denoising. This requires a simple adaptation of classic denoising algorithms to demosaiced noise, which we justify and specify. Although our main conclusion is 'demosaic first, then denoise,' we also discover that for high noise, there is a moderate PSNR gain by a more complex strategy: partial CFA denoising followed by demosaicing and by a second denoising on the RGB image. These surprising results are obtained by a black-box optimization of the pipeline, which could be applied to any other pipeline. We validate our results on simulated and real noisy CFA images obtained from several benchmarks.\",\"PeriodicalId\":50274,\"journal\":{\"name\":\"Inverse Problems and Imaging\",\"volume\":\"19 1\",\"pages\":\"0\"},\"PeriodicalIF\":1.2000,\"publicationDate\":\"2023-01-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Inverse Problems and Imaging\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.3934/ipi.2023044\",\"RegionNum\":4,\"RegionCategory\":\"数学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"MATHEMATICS, APPLIED\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Inverse Problems and Imaging","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.3934/ipi.2023044","RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"MATHEMATICS, APPLIED","Score":null,"Total":0}
引用次数: 0

摘要

图像去马赛克和去噪在原始成像过程中起着至关重要的作用。这些过程通常被视为独立的,而没有考虑它们之间的相互作用。实际上,大多数经典的去噪方法处理有噪声的RGB图像,而不是原始图像。相反,大多数反马赛克方法都是针对无噪声图像的反马赛克。真正的问题是如何对有噪声的原始图像进行去噪和去噪。但是如何进行的问题仍然没有明确。在本文中,我们进行了大量的实验和数学分析,以解决这个问题的低复杂度算法。事实上,这两个问题都只能通过端到端的重权重卷积神经网络(cnn)来解决,而cnn目前与低功耗便携式成像设备不兼容,并且本质上依赖于域(或设备)。我们的研究使我们得出结论,在适度噪声的情况下,应该先进行去马赛克处理,然后再进行去噪处理。这需要一个简单的经典去噪算法的适应去噪噪声,我们证明和说明。虽然我们的主要结论是“首先去噪,然后去噪”,但我们也发现,对于高噪声,通过更复杂的策略可以获得适度的PSNR增益:部分CFA去噪,然后去噪,然后在RGB图像上进行第二次去噪。这些令人惊讶的结果是通过管道的黑盒优化获得的,可以应用于任何其他管道。我们在几个基准测试中获得的模拟和真实的噪声CFA图像上验证了我们的结果。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
How to best combine demosaicing and denoising?
Image demosaicing and denoising play a critical role in the raw imaging pipeline. These processes have often been treated as independent, without considering their interactions. Indeed, most classic denoising methods handle noisy RGB images, not raw images. Conversely, most demosaicing methods address the demosaicing of noise free images. The real problem is to jointly denoise and demosaic noisy raw images. But the question of how to proceed is still not clarified. In this paper, we carry out extensive experiments and a mathematical analysis to tackle this problem by low complexity algorithms. Indeed, both problems have only been addressed jointly by end-to-end heavy-weight convolutional neural networks (CNNs), which are currently incompatible with low-power portable imaging devices and remain by nature domain (or device) dependent. Our study leads us to conclude that, with moderate noise, demosaicing should be applied first, followed by denoising. This requires a simple adaptation of classic denoising algorithms to demosaiced noise, which we justify and specify. Although our main conclusion is 'demosaic first, then denoise,' we also discover that for high noise, there is a moderate PSNR gain by a more complex strategy: partial CFA denoising followed by demosaicing and by a second denoising on the RGB image. These surprising results are obtained by a black-box optimization of the pipeline, which could be applied to any other pipeline. We validate our results on simulated and real noisy CFA images obtained from several benchmarks.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Inverse Problems and Imaging
Inverse Problems and Imaging 数学-物理:数学物理
CiteScore
2.50
自引率
0.00%
发文量
55
审稿时长
>12 weeks
期刊介绍: Inverse Problems and Imaging publishes research articles of the highest quality that employ innovative mathematical and modeling techniques to study inverse and imaging problems arising in engineering and other sciences. Every published paper has a strong mathematical orientation employing methods from such areas as control theory, discrete mathematics, differential geometry, harmonic analysis, functional analysis, integral geometry, mathematical physics, numerical analysis, optimization, partial differential equations, and stochastic and statistical methods. The field of applications includes medical and other imaging, nondestructive testing, geophysical prospection and remote sensing as well as image analysis and image processing. This journal is committed to recording important new results in its field and will maintain the highest standards of innovation and quality. To be published in this journal, a paper must be correct, novel, nontrivial and of interest to a substantial number of researchers and readers.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信