SiamORPN: Enabling Orthogonality between Object and Background in Siamese Object Tracking

Abstract

Siamese-based trackers currently are the dominant tracking paradigm due to the balance between speed and performance. However, it is prone to drift and tracking failure when the environment is complex and similar objects interfere. While the Siamese-based trackers perform the correlation operation, the responses of the target object and background appear in different channels, i.e., the feature spaces of the target object and background have some orthogonality. However, when meeting background clutters and similar objects interfere, this orthogonality becomes weaker and the wrong classification contribution of the object and the background reduces the stability of the learned similarity function, leading to many misclassified pixels in the heatmaps. In this work, we proposed a SiamORPN to solve the above issues. It is incorporated at two levels: an Orthogonal Region Proposal Network (ORPN) and an Adaptive Pixel-wise Aggregation (APA) module. Specifically, for ORPN, the orthogonality between the object and the background maximizes the inter-class inertia. Moreover, the ORPN introduces the orthogonal module to enhance this orthogonality. For APA, it introduces two lightweight networks to predict the weights of all pixels in different heatmaps and the weights of all pixels in different regression offsets. Experiments on challenging benchmarks, including OTB2015, VOT2016, VOT2018, GOT-10k test set, UAV123, LaSOT, and TrackingNet, demonstrate the proposed SiamORPN outperforms many SOTA trackers and achieves leading performance. The inference speed at GTX1080Ti can reach about 32 FPS, meeting the real-time requirements.