Jingyun Liu , Han Zhu , Daiqin Yang , Zhenzhong Chen , Shan Liu
{"title":"Meta-RawResampler:基于模式引导的原始图像重新缩放","authors":"Jingyun Liu , Han Zhu , Daiqin Yang , Zhenzhong Chen , Shan Liu","doi":"10.1016/j.jvcir.2025.104514","DOIUrl":null,"url":null,"abstract":"<div><div>Modern digital cameras allow users to set the resolution at which images are saved. When low-resolution images (LR) are required for storage, images will be downscaled. If high-resolution images (HR) are needed subsequently, upscaling will be performed. Image rescaling aims at jointly optimizing downscaling/upscaling to achieve both visually plausible LR and high-fidelity HR. However, previous works have primarily focused on rescaling from RGB images. They cannot alleviate the errors produced during image signal processing (ISP), particularly arising from demosaicking and denoising. Such errors may propagate through the downscaling process and ultimately degrade the upscaling results. In contrast, we directly produce LR from noisy raw images, facing additional challenges due to incomplete color information and noises in raw images that hinder the encoding of texture details from high-resolution images into their LR counterparts. To this issue, Meta-RawResampler is proposed, which performs downscaling with spatial-color wise resampling kernels. The kernel weights are generated under the guidance of both pattern information and image content to facilitate the interaction between color channels. This interaction helps the model to infer information about missing colors based on other recorded colors, thereby enhancing the network’s ability to understand and further preserve high-frequency information. Moreover, a Pattern-Content Dynamic Guidance Module (PCDG) is proposed, which is decomposed into a Channel-wise Per-pixel Color Interpolation Block and a Color-wise Feature Interpolation Block. The former utilizes pattern information and image content to generate channel-wise spatial adaptive kernel weights, fully exploring color correlation between color, channel, and spatial dimensions to facilitate adaptive color interaction. Meanwhile, the latter employs color-wise convolution to further enhance the model’s ability to learn spatial information. Through these designs, our resampler can achieve upscaling results with higher fidelity. Extensive experiments validate the superiority of the proposed Meta-RawResampler both quantitatively and qualitatively.</div></div>","PeriodicalId":54755,"journal":{"name":"Journal of Visual Communication and Image Representation","volume":"111 ","pages":"Article 104514"},"PeriodicalIF":3.1000,"publicationDate":"2025-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Meta-RawResampler: Raw image rescaling based on pattern guidance\",\"authors\":\"Jingyun Liu , Han Zhu , Daiqin Yang , Zhenzhong Chen , Shan Liu\",\"doi\":\"10.1016/j.jvcir.2025.104514\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Modern digital cameras allow users to set the resolution at which images are saved. When low-resolution images (LR) are required for storage, images will be downscaled. If high-resolution images (HR) are needed subsequently, upscaling will be performed. Image rescaling aims at jointly optimizing downscaling/upscaling to achieve both visually plausible LR and high-fidelity HR. However, previous works have primarily focused on rescaling from RGB images. They cannot alleviate the errors produced during image signal processing (ISP), particularly arising from demosaicking and denoising. Such errors may propagate through the downscaling process and ultimately degrade the upscaling results. In contrast, we directly produce LR from noisy raw images, facing additional challenges due to incomplete color information and noises in raw images that hinder the encoding of texture details from high-resolution images into their LR counterparts. To this issue, Meta-RawResampler is proposed, which performs downscaling with spatial-color wise resampling kernels. The kernel weights are generated under the guidance of both pattern information and image content to facilitate the interaction between color channels. This interaction helps the model to infer information about missing colors based on other recorded colors, thereby enhancing the network’s ability to understand and further preserve high-frequency information. Moreover, a Pattern-Content Dynamic Guidance Module (PCDG) is proposed, which is decomposed into a Channel-wise Per-pixel Color Interpolation Block and a Color-wise Feature Interpolation Block. The former utilizes pattern information and image content to generate channel-wise spatial adaptive kernel weights, fully exploring color correlation between color, channel, and spatial dimensions to facilitate adaptive color interaction. Meanwhile, the latter employs color-wise convolution to further enhance the model’s ability to learn spatial information. Through these designs, our resampler can achieve upscaling results with higher fidelity. Extensive experiments validate the superiority of the proposed Meta-RawResampler both quantitatively and qualitatively.</div></div>\",\"PeriodicalId\":54755,\"journal\":{\"name\":\"Journal of Visual Communication and Image Representation\",\"volume\":\"111 \",\"pages\":\"Article 104514\"},\"PeriodicalIF\":3.1000,\"publicationDate\":\"2025-06-18\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Visual Communication and Image Representation\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1047320325001282\",\"RegionNum\":4,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Visual Communication and Image Representation","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1047320325001282","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
Meta-RawResampler: Raw image rescaling based on pattern guidance
Modern digital cameras allow users to set the resolution at which images are saved. When low-resolution images (LR) are required for storage, images will be downscaled. If high-resolution images (HR) are needed subsequently, upscaling will be performed. Image rescaling aims at jointly optimizing downscaling/upscaling to achieve both visually plausible LR and high-fidelity HR. However, previous works have primarily focused on rescaling from RGB images. They cannot alleviate the errors produced during image signal processing (ISP), particularly arising from demosaicking and denoising. Such errors may propagate through the downscaling process and ultimately degrade the upscaling results. In contrast, we directly produce LR from noisy raw images, facing additional challenges due to incomplete color information and noises in raw images that hinder the encoding of texture details from high-resolution images into their LR counterparts. To this issue, Meta-RawResampler is proposed, which performs downscaling with spatial-color wise resampling kernels. The kernel weights are generated under the guidance of both pattern information and image content to facilitate the interaction between color channels. This interaction helps the model to infer information about missing colors based on other recorded colors, thereby enhancing the network’s ability to understand and further preserve high-frequency information. Moreover, a Pattern-Content Dynamic Guidance Module (PCDG) is proposed, which is decomposed into a Channel-wise Per-pixel Color Interpolation Block and a Color-wise Feature Interpolation Block. The former utilizes pattern information and image content to generate channel-wise spatial adaptive kernel weights, fully exploring color correlation between color, channel, and spatial dimensions to facilitate adaptive color interaction. Meanwhile, the latter employs color-wise convolution to further enhance the model’s ability to learn spatial information. Through these designs, our resampler can achieve upscaling results with higher fidelity. Extensive experiments validate the superiority of the proposed Meta-RawResampler both quantitatively and qualitatively.
期刊介绍:
The Journal of Visual Communication and Image Representation publishes papers on state-of-the-art visual communication and image representation, with emphasis on novel technologies and theoretical work in this multidisciplinary area of pure and applied research. The field of visual communication and image representation is considered in its broadest sense and covers both digital and analog aspects as well as processing and communication in biological visual systems.