An unsupervised video stabilization algorithm based on gyroscope image fusion

Abstract

Video stabilization aims to enhance the visual quality by reducing jitter and ghosting artifacts caused by camera shaking, yet effectively stabilizing low-quality videos and from complex scenarios remains a significant challenge. While gyroscope-based approaches can address this issue, they struggle with depth variations and translational shaking. In this paper, we propose a coarse-to-fine, unsupervised deep learning video stabilization solution that integrates image and gyroscope data to address these challenges. Our approach excels in stabilizing videos under diverse conditions, managing depth changes, and handling both translational and rotational motion. We utilize gyroscope data to estimate the 3D camera rotation and apply LSTM to predict stable poses. Grid-based motion parameters address depth-related motion, generating a multi-grid warping field that mitigates the significant image jitter caused by camera rotation. Subsequently, we achieve the elimination of residual motion at the pixel level. PDCNet is used to generated confidence maps filter optical flow to minimize disturbances from prominent local areas, while a U-Net architecture smooths the optical flow, performing pixel-level warping to generating finely stabilized frames. Comparative analysis shows that our approach surpasses state-of-the-art methods, particularly in handling complex scenes and achieving stability in challenging conditions.