MEFSCFFormer: Multiscale edge-aware fusion block with stereo cross Fourier transformer for stereo image super-resolution and diffusion-based image enhancement

Abstract

Current stereo super-resolution (SR) methods present significant challenges in the effective exploitation of intra-view and inter-view features, especially in how to simultaneously maintain structural coherence and high-frequency detail recovery. To address these challenges, we propose Multiscale Edge-Aware Fusion Block with Stereo Cross Fourier Transformer(MEFSCFFormer) to better utilize intra-view and inter-view information for feature extraction, alignment and fusion. Proposed Multiscale Edge-Aware Fusion Block(MEFB) integrates the Multiscale Edge-Enhanced Mobile Convolution Block Module(MEMB) and the Multi-level Decentralized Mixed Pooled Spatial Attention Module(MDMPSA) to achieve efficient fusion of global and local features, which also combines edge information to better capture structural details that are consistent across viewpoints. To further enhance inter-view information, we design a Stereo Cross Fourier Transformer Module(SCFFormer) that adaptively selects and enhances cross-view-consistent frequency components in stereo images that contribute to the recovery. Besides the MEFSCFFormer can access the Diffusion model and fine-tune the supervised fine-tuning layer to further improve SR subjective quality. This approach overcomes the shortcomings of existing stereo image processing methods in viewpoint-consistent processing and significantly improves the accuracy and detail fidelity of stereo image restoration. We have conducted extensive experiments on several public datasets (Flickr1024 [1], KITTI2012 [2], KITTI2015 [3] and Middlebury [4]). The experimental results show that our method excels in several evaluation metrics compared to other state-of-the-art methods, especially in maintaining a new level of detail accuracy and structural consistency.