Spatio-temporal Feature-level Augmentation Vision Transformer for video-based person re-identification

Abstract

Video-based person re-identification (ReID) aims to match an individual across multiple videos, thus addressing critical aspects of security applications of computer vision. While previous transformer-based approaches have used various means to enhance performance, the growing complexities in network design have posed challenges in meeting the practical requirements of intelligent surveillance systems. To improve network efficiency, we introduce a Feature-level Augmentation Vision Transformer (FAViT), which reinterprets the attributes of video ReID. We leverage the property of maintaining identity even when backgrounds change or multiple persons appear in video frames. First, we introduce Token Representation Learning to distinguish foreground from background. We also employ spatio-temporal feature-level augmentation, along with conducting Altered Background ID classification and Anomaly Frame Detection, to strengthen the representation capacity of the transformer. Extensive experiments validate the effectiveness of FAViT with the least computational overhead among transformer-based models across five benchmarks. We substantiate our model’s generalization ability through analyses.