基于深度信息条件的文本生成引导下的单幅图像脱轨算法

IF 7.2 1区 计算机科学 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE
Xing Wei, Xiufen Ye, Xinkui Mei, Junting Wang, Heming Ma
{"title":"基于深度信息条件的文本生成引导下的单幅图像脱轨算法","authors":"Xing Wei,&nbsp;Xiufen Ye,&nbsp;Xinkui Mei,&nbsp;Junting Wang,&nbsp;Heming Ma","doi":"10.1016/j.asoc.2025.113506","DOIUrl":null,"url":null,"abstract":"<div><div>Currently, image denoising algorithms based on text-to-image diffusion models often encounter issues with disordered internal structure layouts and discrepancies in detail when generating high-resolution images. To address these issues, we proposed a single image deraining algorithm guided by text generation based on depth information conditions. We designed a depth information encoder aimed at leveraging the depth information in rainy images to enhance the spatial mapping between text-to-image and image-to-text, thereby improving the internal structural layout of the generated images. To make the texture details of the generated image domain more similar to those of the original image domain, we designed a Cross Attention module that uses difference information to make the images in both domains more similar, thereby enhancing the guidance of existing deraining algorithms. Experimental results on multiple benchmark datasets demonstrate that the proposed algorithm outperforms state-of-the-art image deraining methods in both visual quality and quantitative performance. On average, it achieves an improvement of 0.46 in SSIM and 0.79 dB in PSNR, effectively removing rain streaks while preserving fine image details and maintaining structural consistency. We will release our code on <span><span>Github</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50737,"journal":{"name":"Applied Soft Computing","volume":"181 ","pages":"Article 113506"},"PeriodicalIF":7.2000,"publicationDate":"2025-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A single image deraining algorithm guided by text generation based on depth information conditions\",\"authors\":\"Xing Wei,&nbsp;Xiufen Ye,&nbsp;Xinkui Mei,&nbsp;Junting Wang,&nbsp;Heming Ma\",\"doi\":\"10.1016/j.asoc.2025.113506\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Currently, image denoising algorithms based on text-to-image diffusion models often encounter issues with disordered internal structure layouts and discrepancies in detail when generating high-resolution images. To address these issues, we proposed a single image deraining algorithm guided by text generation based on depth information conditions. We designed a depth information encoder aimed at leveraging the depth information in rainy images to enhance the spatial mapping between text-to-image and image-to-text, thereby improving the internal structural layout of the generated images. To make the texture details of the generated image domain more similar to those of the original image domain, we designed a Cross Attention module that uses difference information to make the images in both domains more similar, thereby enhancing the guidance of existing deraining algorithms. Experimental results on multiple benchmark datasets demonstrate that the proposed algorithm outperforms state-of-the-art image deraining methods in both visual quality and quantitative performance. On average, it achieves an improvement of 0.46 in SSIM and 0.79 dB in PSNR, effectively removing rain streaks while preserving fine image details and maintaining structural consistency. We will release our code on <span><span>Github</span><svg><path></path></svg></span>.</div></div>\",\"PeriodicalId\":50737,\"journal\":{\"name\":\"Applied Soft Computing\",\"volume\":\"181 \",\"pages\":\"Article 113506\"},\"PeriodicalIF\":7.2000,\"publicationDate\":\"2025-07-03\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Applied Soft Computing\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1568494625008178\",\"RegionNum\":1,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Applied Soft Computing","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1568494625008178","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0

摘要

目前,基于文本到图像扩散模型的图像去噪算法在生成高分辨率图像时,往往会遇到内部结构布局混乱、细节不一致等问题。为了解决这些问题,我们提出了一种基于深度信息条件的文本生成引导的单幅图像脱轨算法。我们设计了一种深度信息编码器,旨在利用降雨图像中的深度信息,增强文本与图像、图像与文本之间的空间映射,从而改善生成图像的内部结构布局。为了使生成图像域的纹理细节与原始图像域的纹理细节更加相似,我们设计了Cross Attention模块,利用差分信息使两个域的图像更加相似,从而增强了现有脱轨算法的导向性。在多个基准数据集上的实验结果表明,该算法在视觉质量和定量性能上都优于目前最先进的图像去噪方法。平均SSIM提高0.46,PSNR提高0.79 dB,在有效去除雨纹的同时,保留了图像的精细细节,保持了结构一致性。我们将在Github上发布我们的代码。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
A single image deraining algorithm guided by text generation based on depth information conditions
Currently, image denoising algorithms based on text-to-image diffusion models often encounter issues with disordered internal structure layouts and discrepancies in detail when generating high-resolution images. To address these issues, we proposed a single image deraining algorithm guided by text generation based on depth information conditions. We designed a depth information encoder aimed at leveraging the depth information in rainy images to enhance the spatial mapping between text-to-image and image-to-text, thereby improving the internal structural layout of the generated images. To make the texture details of the generated image domain more similar to those of the original image domain, we designed a Cross Attention module that uses difference information to make the images in both domains more similar, thereby enhancing the guidance of existing deraining algorithms. Experimental results on multiple benchmark datasets demonstrate that the proposed algorithm outperforms state-of-the-art image deraining methods in both visual quality and quantitative performance. On average, it achieves an improvement of 0.46 in SSIM and 0.79 dB in PSNR, effectively removing rain streaks while preserving fine image details and maintaining structural consistency. We will release our code on Github.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Applied Soft Computing
Applied Soft Computing 工程技术-计算机:跨学科应用
CiteScore
15.80
自引率
6.90%
发文量
874
审稿时长
10.9 months
期刊介绍: Applied Soft Computing is an international journal promoting an integrated view of soft computing to solve real life problems.The focus is to publish the highest quality research in application and convergence of the areas of Fuzzy Logic, Neural Networks, Evolutionary Computing, Rough Sets and other similar techniques to address real world complexities. Applied Soft Computing is a rolling publication: articles are published as soon as the editor-in-chief has accepted them. Therefore, the web site will continuously be updated with new articles and the publication time will be short.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信