Hyper-Feature Based Tracking with the Fully-Convolutional Siamese Network

Abstract

Convolutional neural network (CNN) has drawn increasing interest in visual tracking, among which fully-convolutional Siamese network based method (SiamFC) is quite popular due to its competitive performance in both precision and efficiency. Generally, SiamFC captures robust semantics from high-level features in the last layer but ignores detailed spatial features in earlier layers, thus tending to drift towards similar target regions in the search area. In this paper, we design a skip-layer connection network on the basis of SiamFC to aggregate hierarchical feature maps and constitute the hyper- feature representations of target, considering that convolutional layers in different levels characterize target from different perspectives and the lower-level feature maps of SiamFC is computed beforehand. The Hyper-features well incorporate deep but highly semantic, intermediate but really complementary, and shallow but naturally high-resolution representations. The designed network is trained end-to-end offline similar to SiamFC on the ILSVRC2015 dataset and later used for online tracking. Experimental results on OTB benchmark show that the proposed algorithm performs favourably against many state-of-the-art trackers in terms of accuracy while maintaining real-time tracking speed.