A robust and adaptive framework with space–time memory networks for Visual Object Tracking

Abstract

These trackers based on the space–time memory network locate the target object in the search image employing contextual information from multiple memory frames and their corresponding foreground–background features. It is conceivable that these trackers are susceptible to the memory frame quality as well as the accuracy of the corresponding foreground labels. In the previous works, experienced methods are employed to obtain memory frames from historical frames, which hinders the improvement of generalization and performance. To address the above limitations, we propose a robust and adaptive extraction strategy for memory frames to ensure that the most representative historical frames are selected into the set of memory frames to increase the accuracy of localization and reduce failures due to error accumulation. Specifically, we propose an extraction network to evaluate historical frames, where historical frames with the highest score are labeled as the memory frame and conversely dropped. Qualitative and quantitative analyses were implemented on multiple datasets (OTB100, LaSOT and GOT-10K), and the proposed method obtains significant gain in performance over the previous works, especially for challenging scenarios. while bringing only a negligible inference speed degradation, otherwise, the proposed method obtains competitive results compared to other state-of-the-art (SOTA) methods.