End-to-end video object detection based on dynamic anchor box spatiotemporal decoder and hybrid matching

Abstract

Despite the significant progress in object detection using single-frame images, when dealing with continuously changing video scenes, relying solely on information from individual frames often fails to fully exploit the dynamic continuity and consistency across the temporal dimension. Designing a model structure that can capture local details while also understanding the global spatiotemporal context remains a challenging problem. To address this, this paper proposes an end-to-end video object detection algorithm based on dynamic anchor box spatiotemporal decoder and hybrid matching (DAHM-Net). Specifically, a dynamic anchor box spatiotemporal decoder is proposed, where each decoder layer iteratively updates the position priors that account for object scales, enabling the model to better learn object features and improve detection performance. Additionally, a hybrid matching training strategy is proposed, combining one-to-one and one-to-many matching to enhance model performance and accelerate convergence, while maintaining end-to-end detection. Experimental results show that the proposed method outperforms TransVOD by 0.7, 1.0, and 0.5 AP₅₀ on three public datasets, and also surpasses recent state-of-the-art approaches.