用于高噪声图像去噪的先进变压器:增强关注和细节保存

IF 3.4 3区 工程技术 Q2 ENGINEERING, ELECTRICAL & ELECTRONIC
Jie Zhang , Wenxiao Huang , Miaoxin Lu , Fengxian Wang , Mingdong Zhao , Yinhua Li
{"title":"用于高噪声图像去噪的先进变压器:增强关注和细节保存","authors":"Jie Zhang ,&nbsp;Wenxiao Huang ,&nbsp;Miaoxin Lu ,&nbsp;Fengxian Wang ,&nbsp;Mingdong Zhao ,&nbsp;Yinhua Li","doi":"10.1016/j.image.2025.117286","DOIUrl":null,"url":null,"abstract":"<div><div>In image denoising, the transformer model effectively captures global dependencies within an image due to its self-attention mechanism. This capability enhances the understanding of the overall structure and details of the image during the denoising process. However, the computational complexity of global self-attention increases quadratically with higher spatial resolutions, making it unsuitable for the real-time denoising of high-resolution and high-noise images. And, the use of local windows alone neglects the long-range pixel correlations. Furthermore, the self-attention mechanism applies a global weighting to the pixels of the input image, which can lead to the smoothing or loss of fine details. To enrich structural information and alleviate the computational complexity associated with global self-attention, we propose an edge-enhanced windowed multi-head self-attention mechanism (EWMSA). This mechanism combines edge enhancement with windowed self-attention to reduce computational demands while allowing edge features to better preserve detail and texture information. To mitigate the effects of ineffective features with low weights, we introduce a feed-forward network with a gate control strategy (LGFN). This network adjusts pixel weights to prioritize attention on effective pixels, thereby enhancing their prominence. Furthermore, to compensate for the limitations of window-based self-attention in global pixel utilization, we propose a deformable convolution block (DFCB). This block improves the interaction of contextual information and allows for better adaptation to texture variations within the image. Extensive experiments demonstrate that the proposed ATHID is competitive with other state-of-the-art denoising methods when applied to real-world noise and various synthetic noise levels, effectively addressing the challenges of high-noise image denoising. The code and models are publicly available at <span><span>https://github.com/zzuli407/ATHID</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49521,"journal":{"name":"Signal Processing-Image Communication","volume":"136 ","pages":"Article 117286"},"PeriodicalIF":3.4000,"publicationDate":"2025-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Advanced transformer for high-noise image denoising: Enhanced attention and detail preservation\",\"authors\":\"Jie Zhang ,&nbsp;Wenxiao Huang ,&nbsp;Miaoxin Lu ,&nbsp;Fengxian Wang ,&nbsp;Mingdong Zhao ,&nbsp;Yinhua Li\",\"doi\":\"10.1016/j.image.2025.117286\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>In image denoising, the transformer model effectively captures global dependencies within an image due to its self-attention mechanism. This capability enhances the understanding of the overall structure and details of the image during the denoising process. However, the computational complexity of global self-attention increases quadratically with higher spatial resolutions, making it unsuitable for the real-time denoising of high-resolution and high-noise images. And, the use of local windows alone neglects the long-range pixel correlations. Furthermore, the self-attention mechanism applies a global weighting to the pixels of the input image, which can lead to the smoothing or loss of fine details. To enrich structural information and alleviate the computational complexity associated with global self-attention, we propose an edge-enhanced windowed multi-head self-attention mechanism (EWMSA). This mechanism combines edge enhancement with windowed self-attention to reduce computational demands while allowing edge features to better preserve detail and texture information. To mitigate the effects of ineffective features with low weights, we introduce a feed-forward network with a gate control strategy (LGFN). This network adjusts pixel weights to prioritize attention on effective pixels, thereby enhancing their prominence. Furthermore, to compensate for the limitations of window-based self-attention in global pixel utilization, we propose a deformable convolution block (DFCB). This block improves the interaction of contextual information and allows for better adaptation to texture variations within the image. Extensive experiments demonstrate that the proposed ATHID is competitive with other state-of-the-art denoising methods when applied to real-world noise and various synthetic noise levels, effectively addressing the challenges of high-noise image denoising. The code and models are publicly available at <span><span>https://github.com/zzuli407/ATHID</span><svg><path></path></svg></span>.</div></div>\",\"PeriodicalId\":49521,\"journal\":{\"name\":\"Signal Processing-Image Communication\",\"volume\":\"136 \",\"pages\":\"Article 117286\"},\"PeriodicalIF\":3.4000,\"publicationDate\":\"2025-02-26\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Signal Processing-Image Communication\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0923596525000335\",\"RegionNum\":3,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Signal Processing-Image Communication","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0923596525000335","RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0

摘要

在图像去噪中,由于变压器模型的自关注机制,它可以有效地捕获图像中的全局依赖关系。这种能力增强了在去噪过程中对图像整体结构和细节的理解。然而,随着空间分辨率的提高,全局自关注的计算复杂度呈二次增长,不适合高分辨率和高噪声图像的实时去噪。而且,单独使用局部窗口忽略了远程像素相关性。此外,自注意机制对输入图像的像素应用全局加权,这可能导致平滑或精细细节的丢失。为了丰富结构信息,降低全局自注意的计算复杂度,提出了一种边缘增强窗口多头自注意机制(EWMSA)。该机制将边缘增强与窗口自关注相结合,以减少计算需求,同时允许边缘特征更好地保留细节和纹理信息。为了减轻低权重无效特征的影响,我们引入了一个带有门控制策略(LGFN)的前馈网络。该网络调整像素权重,优先关注有效像素,从而增强其突出性。此外,为了弥补基于窗口的自关注在全局像素利用中的局限性,我们提出了一种可变形卷积块(DFCB)。该块改善了上下文信息的交互,并允许更好地适应图像中的纹理变化。大量实验表明,当应用于实际噪声和各种合成噪声水平时,所提出的ATHID与其他最先进的去噪方法相比具有竞争力,有效地解决了高噪声图像去噪的挑战。代码和模型可在https://github.com/zzuli407/ATHID上公开获得。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Advanced transformer for high-noise image denoising: Enhanced attention and detail preservation
In image denoising, the transformer model effectively captures global dependencies within an image due to its self-attention mechanism. This capability enhances the understanding of the overall structure and details of the image during the denoising process. However, the computational complexity of global self-attention increases quadratically with higher spatial resolutions, making it unsuitable for the real-time denoising of high-resolution and high-noise images. And, the use of local windows alone neglects the long-range pixel correlations. Furthermore, the self-attention mechanism applies a global weighting to the pixels of the input image, which can lead to the smoothing or loss of fine details. To enrich structural information and alleviate the computational complexity associated with global self-attention, we propose an edge-enhanced windowed multi-head self-attention mechanism (EWMSA). This mechanism combines edge enhancement with windowed self-attention to reduce computational demands while allowing edge features to better preserve detail and texture information. To mitigate the effects of ineffective features with low weights, we introduce a feed-forward network with a gate control strategy (LGFN). This network adjusts pixel weights to prioritize attention on effective pixels, thereby enhancing their prominence. Furthermore, to compensate for the limitations of window-based self-attention in global pixel utilization, we propose a deformable convolution block (DFCB). This block improves the interaction of contextual information and allows for better adaptation to texture variations within the image. Extensive experiments demonstrate that the proposed ATHID is competitive with other state-of-the-art denoising methods when applied to real-world noise and various synthetic noise levels, effectively addressing the challenges of high-noise image denoising. The code and models are publicly available at https://github.com/zzuli407/ATHID.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Signal Processing-Image Communication
Signal Processing-Image Communication 工程技术-工程:电子与电气
CiteScore
8.40
自引率
2.90%
发文量
138
审稿时长
5.2 months
期刊介绍: Signal Processing: Image Communication is an international journal for the development of the theory and practice of image communication. Its primary objectives are the following: To present a forum for the advancement of theory and practice of image communication. To stimulate cross-fertilization between areas similar in nature which have traditionally been separated, for example, various aspects of visual communications and information systems. To contribute to a rapid information exchange between the industrial and academic environments. The editorial policy and the technical content of the journal are the responsibility of the Editor-in-Chief, the Area Editors and the Advisory Editors. The Journal is self-supporting from subscription income and contains a minimum amount of advertisements. Advertisements are subject to the prior approval of the Editor-in-Chief. The journal welcomes contributions from every country in the world. Signal Processing: Image Communication publishes articles relating to aspects of the design, implementation and use of image communication systems. The journal features original research work, tutorial and review articles, and accounts of practical developments. Subjects of interest include image/video coding, 3D video representations and compression, 3D graphics and animation compression, HDTV and 3DTV systems, video adaptation, video over IP, peer-to-peer video networking, interactive visual communication, multi-user video conferencing, wireless video broadcasting and communication, visual surveillance, 2D and 3D image/video quality measures, pre/post processing, video restoration and super-resolution, multi-camera video analysis, motion analysis, content-based image/video indexing and retrieval, face and gesture processing, video synthesis, 2D and 3D image/video acquisition and display technologies, architectures for image/video processing and communication.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信