Frequency-aware fusion for improved video object segmentation

Currently, most mainstream memory-based semi-supervised video object segmentation (VOS) methods rely on pixel-level matching to identify target objects. However, the majority of these approaches depend solely on spatial-domain features for representation, which limits their ability to preserve fine-grained details. In addition, they typically adopt a single bottom-up matching strategy, which lacks sufficient global semantic guidance, ultimately leading to suboptimal segmentation performance. To address these issues, we propose a Frequency-Aware Fusion for Improved Video Object Segmentation algorithm (FAFVOS), which incorporates frequency-domain information enhancement and a bidirectional matching mechanism to improve segmentation accuracy. First, we design a Hierarchical Frequency-Aware Encoder (HFAE), which enhances shallow features by leveraging high-frequency components to preserve edge and texture details, and strengthens deep features via low-frequency components to maintain global structural consistency, thereby achieving multi-scale frequency–spatial feature fusion. Second, a frequency-guided bidirectional matching Transformer module is proposed to establish pixel-level and object-level dual-path interactions. By incorporating a cross-attention mechanism, the model effectively facilitates joint reasoning between local pixel-wise details and global object-level semantics. Finally, a high-order moment refinement module is introduced to integrate high-order statistical features, enhancing the model’s ability to capture object deformation and leading to high-quality segmentation results. The proposed method is evaluated on the DAVIS, YouTube-VOS, and MOSE datasets. Experimental results demonstrate that, without relying on complex pretraining strategies or additional datasets, our approach achieves a real-time inference speed of 56 FPS with a $J\&F$ score of 88.5 % on the DAVIS 2017 benchmark, surpassing existing representative methods. Moreover, it also achieves consistently superior performance on the more challenging YouTube-VOS and MOSE datasets, further validating the generalization ability and robustness of the proposed approach.