Enhancing feature representation in Siamese networks for object tracking with ranking-based loss

Abstract

Siamese networks have gained significant traction in single-object tracking due to their efficient feature comparison and matching capabilities. However, existing approaches often struggle to capture robust and discriminative representations under complex conditions such as occlusion, scale variation, and background clutter, leading to a trade-off between classification accuracy and localization precision. To overcome these limitations, this paper introduces SiamMFFRL, a novel Siamese-based tracking framework that integrates multi-feature fusion and ranking-based loss learning to achieve balanced and robust target estimation. The proposed multi-feature fusion module (MFF) aggregates hierarchical features from multiple levels of the backbone network, enhancing the discriminative capability of the model and improving its robustness against appearance variations and background distractions. We design a ranking-based loss that jointly optimizes classification and regression branches by enforcing a ranking margin between foreground and background responses, thereby harmonizing classification confidence with localization precision. Furthermore, a target-aware branch (TAB) is incorporated to refine bounding box predictions and mitigate inconsistencies between classification and localization outputs. By integrating these complementary components into a unified Siamese framework, SiamMFFRL achieves a robust balance between accuracy and stability in complex tracking environments. Extensive experiments conducted on major tracking benchmarks, including LaSOT, GOT-10K, VOT2018, VOT2019, OTB100, and UAV123, demonstrate that SiamMFFRL consistently outperforms state-of-the-art trackers in both precision and success rates, confirming its effectiveness and generalization capability.