Video Wire Inpainting via Hierarchical Feature Mixture

Abstract

Video wire inpainting aims at automatically eliminating visible wires from film footage, significantly streamlining post-production workflows. Previous models address redundancy in wire removal by eliminating redundant blocks to enhance focus on crucial wire details for more accurate reconstruction. However, once redundancy is removed, the disorganized non-redundant blocks disrupt temporal and spatial coherence, making seamless inpainting challenging. The absence of multi-scale feature fusion further limits the model’s ability to handle different wire scales and blend inpainted regions with complex backgrounds. To address these challenges, we propose a Hierarchical Feature Mixture Network (HFM-Net) that integrates two novel modules: a Hierarchical Transformer Module (HTM) and a Spatio-temporal Feature Mixture Module (SFM). Specifically, the HTM employs redundancy-aware attention modules and lightweight transformers to reorganize and fuse key high- and low-dimensional patches. The lightweight transformers are sufficient due to the reduced number of non-redundant blocks processing. By aggregating similar features, these transformers guide the alignment of non-redundant blocks and achieve effective spatio-temporal synchronization. Building on this, the SFM incorporates gated convolutions and GRU to enhance spatial and temporal integration further. Gated convolutions fuse low- and high-dimensional features, while the GRU captures temporal dependencies, enabling seamless inpainting of dynamic wire patterns. Additionally, we introduce a lightweight 3D separable convolution discriminator to improve video quality during the inpainting process while reducing computational costs. Experimental results demonstrate that HFM-Net achieves state-of-the-art performance on the video wire removal task.