{"title":"Learning Many-to-Many Mapping for Unpaired Real-World Image Super-resolution and Downscaling.","authors":"Wanjie Sun, Zhenzhong Chen","doi":"10.1109/TPAMI.2024.3428546","DOIUrl":null,"url":null,"abstract":"<p><p>Learning based single image super-resolution (SISR) for real-world images has been an active research topic yet a challenging task, due to the lack of paired low-resolution (LR) and high-resolution (HR) training images. Most of the existing unsupervised real-world SISR methods adopt a twostage training strategy by synthesizing realistic LR images from their HR counterparts first, then training the super-resolution (SR) models in a supervised manner. However, the training of image degradation and SR models in this strategy are separate, ignoring the inherent mutual dependency between downscaling and its inverse upscaling process. Additionally, the ill-posed nature of image degradation is not fully considered. In this paper, we propose an image downscaling and SR model dubbed as SDFlow, which simultaneously learns a bidirectional manyto- many mapping between real-world LR and HR images unsupervisedly. The main idea of SDFlow is to decouple image content and degradation information in the latent space, where content information distribution of LR and HR images is matched in a common latent space. Degradation information of the LR images and the high-frequency information of the HR images are fitted to an easy-to-sample conditional distribution. Experimental results on real-world image SR datasets indicate that SDFlow can generate diverse realistic LR and SR images both quantitatively and qualitatively.</p>","PeriodicalId":94034,"journal":{"name":"IEEE transactions on pattern analysis and machine intelligence","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-07-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE transactions on pattern analysis and machine intelligence","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/TPAMI.2024.3428546","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
Abstract
Learning based single image super-resolution (SISR) for real-world images has been an active research topic yet a challenging task, due to the lack of paired low-resolution (LR) and high-resolution (HR) training images. Most of the existing unsupervised real-world SISR methods adopt a twostage training strategy by synthesizing realistic LR images from their HR counterparts first, then training the super-resolution (SR) models in a supervised manner. However, the training of image degradation and SR models in this strategy are separate, ignoring the inherent mutual dependency between downscaling and its inverse upscaling process. Additionally, the ill-posed nature of image degradation is not fully considered. In this paper, we propose an image downscaling and SR model dubbed as SDFlow, which simultaneously learns a bidirectional manyto- many mapping between real-world LR and HR images unsupervisedly. The main idea of SDFlow is to decouple image content and degradation information in the latent space, where content information distribution of LR and HR images is matched in a common latent space. Degradation information of the LR images and the high-frequency information of the HR images are fitted to an easy-to-sample conditional distribution. Experimental results on real-world image SR datasets indicate that SDFlow can generate diverse realistic LR and SR images both quantitatively and qualitatively.
由于缺乏成对的低分辨率(LR)和高分辨率(HR)训练图像,基于学习的真实世界图像单图像超分辨率(SISR)一直是一个活跃的研究课题,但也是一项具有挑战性的任务。大多数现有的无监督真实世界 SISR 方法都采用了两阶段训练策略,即首先从对应的高分辨率图像中合成真实的低分辨率图像,然后以监督方式训练超分辨率(SR)模型。然而,在这种策略中,图像降级和 SR 模型的训练是分开的,忽略了降级和反向升维过程之间固有的相互依赖性。此外,也没有充分考虑到图像降解的不确定性。在本文中,我们提出了一种被称为 SDFlow 的图像降尺度和升尺度模型,该模型可以在无监督的情况下同时学习真实世界 LR 和 HR 图像之间的双向多对多映射。SDFlow 的主要思想是在潜空间中解耦图像内容和降级信息,即在一个共同的潜空间中匹配 LR 和 HR 图像的内容信息分布。LR 图像的降解信息和 HR 图像的高频信息被拟合到一个易于采样的条件分布中。在真实世界图像 SR 数据集上的实验结果表明,SDFlow 可以定量和定性地生成多种逼真的 LR 和 SR 图像。