Positive Sample Mining: Fuzzy Threshold-Based Contrastive Learning for Enhanced Unsupervised Skeleton-Based Action Recognition

Abstract

Contrastive learning is one of the fundamental paradigms for unsupervised 3-D skeleton-based action recognition. Existing contrastive learning paradigms typically enhance model discrimination by increasing the distance between different action samples in the feature space. However, this approach can inadvertently lead to an increase in the intraclass distance for the same action category, thereby affecting the effectiveness of action recognition. To address this issue, we introduce an innovative unsupervised framework named fuzzy threshold-based contrastive learning (FTCL). This novel approach leverages the concept of fuzzy thresholds to handle sample partitioning within the feature space. In essence, given a dataset of human actions, we distinguish different action samples as “negative samples” and identical action samples as “positive samples.” By analyzing the similarity distribution between these positive and negative samples, we apply the principles of fuzzy thresholds to evaluate the attributes of the negative samples. This refined evaluation facilitates a judicious reassignment of positive and negative sample classifications, thus circumventing the challenges associated with increased intraclass distances. Furthermore, to obtain better action representations from skeleton data, we model and contrast skeleton data from different spatiotemporal perspectives, capturing rich spatiotemporal information in the feature representation of actions. Extensive experiments on the NTU-60, NTU-120, and PKU-MMD datasets were conducted to validate our proposed FTCL. The experimental results demonstrate that our approach achieves significant improvements.