Deep image stitching for panoramic stereoscopic live broadcast system

Image stitching is a crucial task in computer vision, enabling the creation of panoramic images for augmented reality/virtual reality (AR/VR) experiences. Image registration plays an important role in achieving spatial continuity and consistency in image stitching. Therefore, achieving accurate image registration is critical to high-quality panoramic image generation. Although recent learning-based methods have improved the optimization of stitched images, they still rely on simple associations of images or feature maps to search for feature correspondences, neglecting the capture of these correspondences during feature extraction. Additionally, directly warping the original image with the homography matrix extracted from the feature space ignores the network’s influence on image transformation relationships. To address these limitations and enhance the quality of panoramic images, we propose an artificial intelligence (AI) image registration network based on a cross-attention mechanism. Our approach incorporates a Local Transformer to help the network perceive image correspondences. Furthermore, we impose constraints on network training through warp-equivariance to mitigate the impact on image transformation relationships. These strategies increase the accuracy and generalization of the AI-based registration method and form a better image stitching algorithm. Finally, we apply the AI image stitching algorithm to construct a panoramic stereoscopic live broadcast system. Experimental results show that our method achieves competitive results and satisfies the requirements of panoramic systems.