Learning discriminative features via deep metric learning for video-based person re-identification

Video-based person re-identification (Video-Re-ID) is a crucial application in practical scenarios and has gradually emerged as one of the most popular research topics in the field of computer vision. Although many efforts have been made, it still remains a challenge due to substantial variations among person videos and even within each video. For Video-Re-ID, we propose a synchronous intrA-video and intEr-video distance metric learning approach based on temporal ViT architecture, termed as TAE-ViT. The TAE-ViT model, in particular consists of a View Guide Patch Embedding (VGPE) module, a Spatial-Temporal Attention (STA) module and an Intra and Inter-video Distance Metric Learning (IIDML) module. The VGPE module is used to utilize diverse view information and extract discriminative features with perspective invariance. The STA module alternately learns the spatial and temporal information by using the spatial and temporal multi-head attention operation, respectively. The IIDML module simultaneously captures intra-video and inter-video distance metrics from the training videos. Specifically, the intra-video distance metric aims to compact each video representation, while the inter-video distance metric ensures that truly matched videos are closer in distance compared to incorrectly matched ones. Experimental results show that our method achieves the best mAP (86.7 %, 96.3 %, 97.3 %) on three public Video-ReID datasets and achieves the best Rank-1 (93.3 %, 96.6 %) on two datasets, reducing the error of state-of-the-art methods by 1.48-45.16 % on mAP and by 2.86 % on Rank-1. Its robust performance highlights its potential for real-world applications like intelligent surveillance and public safety. Our code will be available at https://github.com/JingShenZhuangTaiYiChang/TAE-ViT.