MMCATrack: Multi-Modal Channel Attention Tracker

Abstract

Most existing Transformer-based visual object tracking methods rely exclusively on the feature map from the last encoder layer for object prediction, thereby overlooking the rich information contained in shallow and intermediate layer feature maps. This limitation reduces the representational capacity of the model. Moreover, current multi-modal tracking frameworks typically construct multi-modal features through simple concatenation, which fails to adequately account for the differential contributions of individual modalities to the final prediction task. As a result, these approaches exhibit an insufficient ability to express key features within the multi-modal representation. To address the aforementioned issues, this paper proposes a multi-modal channel attention tracking algorithm, where a multi-modal channel attention block is incorporated for the purpose of enhancing the representation ability of the key features within the multi-modal features. Specifically, the multi-modal channel attention block first aggregates multi-modal information from the multi-layer feature maps of the encoder through cross layer cascading and then applies channel attention mechanism to dynamically calibrate the channel weights in the generated multi-modal features, thereby enhancing the representation of key features. In addition, this article proposes a new regression loss function to improve localisation accuracy. Finally, abundant experiments conducted on five benchmarks including GOT-10K, TrackingNet, TNL2K, VisEvent and RGBT234 have verified the effectiveness of our theory.