{"title":"当引导扩散模型遇上零镜头图像超分辨率","authors":"","doi":"10.1016/j.engappai.2024.109336","DOIUrl":null,"url":null,"abstract":"<div><p>Existing deep learning-based single-image super-resolution (SR) methods typically rely on vast quantities of paired data. As an essential solution, zero-shot SR methods require only a single image to handle image-specific degradation. However, these methods still struggle to recover fine-grained details due to the lack of supervised information. In this work, we propose a novel guided <strong>Diff</strong>usion model for <strong>Zero</strong>-shot image SR (<strong>ZeroDiff</strong>) to explicitly direct image quality enhancement. Specifically, we elaborate two key guidance strategies: (1) high-frequency guidance and (2) content-consistent guidance. The former concentrates on boosting fine-grained textures by embedding high-frequency information into the cross-attention mechanism of the noise estimator. The latter avoids the sampling deviating from the original image in terms of structure and low-frequency content. Specifically, the noisy images at each diffusion step are injected into the corresponding sampling step, encouraging the sampled image to be consistent with that of the corresponding diffusion step. Moreover, we design a progressive zoom-in paradigm by gradually enlarging the image size and enriching the image details to boost the sampling efficiency of diffusion models, while enabling high-quality image reconstruction. Extensive experiments reveal that our method achieves comparable results with other state-of-the-art methods in quantitative and qualitative evaluations on both face and natural images from synthetic and real-world datasets.</p></div>","PeriodicalId":50523,"journal":{"name":"Engineering Applications of Artificial Intelligence","volume":null,"pages":null},"PeriodicalIF":7.5000,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"When guided diffusion model meets zero-shot image super-resolution\",\"authors\":\"\",\"doi\":\"10.1016/j.engappai.2024.109336\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>Existing deep learning-based single-image super-resolution (SR) methods typically rely on vast quantities of paired data. As an essential solution, zero-shot SR methods require only a single image to handle image-specific degradation. However, these methods still struggle to recover fine-grained details due to the lack of supervised information. In this work, we propose a novel guided <strong>Diff</strong>usion model for <strong>Zero</strong>-shot image SR (<strong>ZeroDiff</strong>) to explicitly direct image quality enhancement. Specifically, we elaborate two key guidance strategies: (1) high-frequency guidance and (2) content-consistent guidance. The former concentrates on boosting fine-grained textures by embedding high-frequency information into the cross-attention mechanism of the noise estimator. The latter avoids the sampling deviating from the original image in terms of structure and low-frequency content. Specifically, the noisy images at each diffusion step are injected into the corresponding sampling step, encouraging the sampled image to be consistent with that of the corresponding diffusion step. Moreover, we design a progressive zoom-in paradigm by gradually enlarging the image size and enriching the image details to boost the sampling efficiency of diffusion models, while enabling high-quality image reconstruction. Extensive experiments reveal that our method achieves comparable results with other state-of-the-art methods in quantitative and qualitative evaluations on both face and natural images from synthetic and real-world datasets.</p></div>\",\"PeriodicalId\":50523,\"journal\":{\"name\":\"Engineering Applications of Artificial Intelligence\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":7.5000,\"publicationDate\":\"2024-09-17\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Engineering Applications of Artificial Intelligence\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0952197624014945\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"AUTOMATION & CONTROL SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Engineering Applications of Artificial Intelligence","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0952197624014945","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
引用次数: 0
摘要
现有的基于深度学习的单图像超分辨率(SR)方法通常依赖于大量的配对数据。作为一种重要的解决方案,零镜头 SR 方法只需要单张图像即可处理特定图像的退化。然而,由于缺乏监督信息,这些方法仍难以恢复细粒度细节。在这项工作中,我们提出了一种新颖的零镜头图像 SR(ZeroDiff)指导性扩散模型,以明确指导图像质量增强。具体来说,我们阐述了两种关键的引导策略:(1) 高频引导和 (2) 内容一致引导。前者通过将高频信息嵌入噪声估计器的交叉注意机制,集中提升细粒度纹理。后者则避免采样偏离原始图像的结构和低频内容。具体来说,每个扩散步骤的噪声图像都会被注入相应的采样步骤,从而促使采样图像与相应扩散步骤的图像保持一致。此外,我们还设计了一种渐进式放大范例,通过逐步放大图像尺寸和丰富图像细节来提高扩散模型的采样效率,同时实现高质量的图像重建。广泛的实验表明,我们的方法在合成和真实世界数据集的人脸图像和自然图像的定量和定性评估中取得了与其他一流方法相当的结果。
When guided diffusion model meets zero-shot image super-resolution
Existing deep learning-based single-image super-resolution (SR) methods typically rely on vast quantities of paired data. As an essential solution, zero-shot SR methods require only a single image to handle image-specific degradation. However, these methods still struggle to recover fine-grained details due to the lack of supervised information. In this work, we propose a novel guided Diffusion model for Zero-shot image SR (ZeroDiff) to explicitly direct image quality enhancement. Specifically, we elaborate two key guidance strategies: (1) high-frequency guidance and (2) content-consistent guidance. The former concentrates on boosting fine-grained textures by embedding high-frequency information into the cross-attention mechanism of the noise estimator. The latter avoids the sampling deviating from the original image in terms of structure and low-frequency content. Specifically, the noisy images at each diffusion step are injected into the corresponding sampling step, encouraging the sampled image to be consistent with that of the corresponding diffusion step. Moreover, we design a progressive zoom-in paradigm by gradually enlarging the image size and enriching the image details to boost the sampling efficiency of diffusion models, while enabling high-quality image reconstruction. Extensive experiments reveal that our method achieves comparable results with other state-of-the-art methods in quantitative and qualitative evaluations on both face and natural images from synthetic and real-world datasets.
期刊介绍:
Artificial Intelligence (AI) is pivotal in driving the fourth industrial revolution, witnessing remarkable advancements across various machine learning methodologies. AI techniques have become indispensable tools for practicing engineers, enabling them to tackle previously insurmountable challenges. Engineering Applications of Artificial Intelligence serves as a global platform for the swift dissemination of research elucidating the practical application of AI methods across all engineering disciplines. Submitted papers are expected to present novel aspects of AI utilized in real-world engineering applications, validated using publicly available datasets to ensure the replicability of research outcomes. Join us in exploring the transformative potential of AI in engineering.