Corner selection and dual network blender for efficient view synthesis in outdoor scenes

Abstract

Novel view synthesis (NVS) from freely distributed viewpoints from large-scale outdoor scenes offers a compelling user experience in several applications, including first-person hyper-lapse videos and virtual reality. However, high-quality NVS requires dense input images, which can be affected by color discrepancies caused by extreme brightness changes, varying viewing angles, incorrect estimation of camera parameters, and mobile objects. This paper introduces Competent View Synthesis (CVS), a cost-effective approach to generating high-quality NVS from large-scale outdoor scenes to address these challenges. CVS employs a three-stage pipeline, including a Corners Selection Algorithm (CSA) to reduce the number of required input images, a Tinkering mechanism to fill in missing pixel data, and a dual-network blending (DNB) model to fuse colors and calculate attention coefficients for feature refinement. The experimental results demonstrate the effectiveness of the proposed CVS in generating realistic viewpoints from a limited number of input viewpoints. Furthermore, the comparative evaluations against two baselines using metrics such as PSNR, SSIM, and LPIPS reveal significant performance improvements of 4.6%, 0.18%, and 30.13%, respectively, over the first baseline, while the proposed method significantly outperforms the second baseline in term of perceptual metrics. By optimizing the synthesis process for complex outdoor scenes, CVS enhances the quality of generated images and improves computational efficiency.