{"title":"EnsCLR:通过表征的集合对比学习实现基于骨骼的无监督动作识别","authors":"","doi":"10.1016/j.cviu.2024.104076","DOIUrl":null,"url":null,"abstract":"<div><p>Skeleton-based action recognition is a key research area in video understanding, beneficial from its compact and efficient motion information. To relieve from the burden of expensive and laborious data annotation, unsupervised approaches, particularly contrastive learning, have been widely employed to extract action representations from unlabeled data. In this paper, we propose an Ensemble framework for Contrastive Learning of Representation (EnsCLR) to preform unsupervised skeleton-based action recognition. Concretely, Queue Extension method is devised to generate discriminative representation by aggregating the ensemble information from multiple pipelines. Furtherly, Ensemble Nearest Neighbors Mining (ENNM) method is utilized to excavate the most similar samples from the unlabeled data as positive samples, which alleviates the false-negative samples problem caused by the disregard of category label. The experiments with extensive evaluation protocols show that EnsCLR outperforms previous state-of-the-art methods on NTU60, NTU120, and PKU-MMD datasets.</p></div>","PeriodicalId":50633,"journal":{"name":"Computer Vision and Image Understanding","volume":null,"pages":null},"PeriodicalIF":4.3000,"publicationDate":"2024-07-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"EnsCLR: Unsupervised skeleton-based action recognition via ensemble contrastive learning of representation\",\"authors\":\"\",\"doi\":\"10.1016/j.cviu.2024.104076\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>Skeleton-based action recognition is a key research area in video understanding, beneficial from its compact and efficient motion information. To relieve from the burden of expensive and laborious data annotation, unsupervised approaches, particularly contrastive learning, have been widely employed to extract action representations from unlabeled data. In this paper, we propose an Ensemble framework for Contrastive Learning of Representation (EnsCLR) to preform unsupervised skeleton-based action recognition. Concretely, Queue Extension method is devised to generate discriminative representation by aggregating the ensemble information from multiple pipelines. Furtherly, Ensemble Nearest Neighbors Mining (ENNM) method is utilized to excavate the most similar samples from the unlabeled data as positive samples, which alleviates the false-negative samples problem caused by the disregard of category label. The experiments with extensive evaluation protocols show that EnsCLR outperforms previous state-of-the-art methods on NTU60, NTU120, and PKU-MMD datasets.</p></div>\",\"PeriodicalId\":50633,\"journal\":{\"name\":\"Computer Vision and Image Understanding\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":4.3000,\"publicationDate\":\"2024-07-14\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Computer Vision and Image Understanding\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1077314224001577\",\"RegionNum\":3,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computer Vision and Image Understanding","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1077314224001577","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
EnsCLR: Unsupervised skeleton-based action recognition via ensemble contrastive learning of representation
Skeleton-based action recognition is a key research area in video understanding, beneficial from its compact and efficient motion information. To relieve from the burden of expensive and laborious data annotation, unsupervised approaches, particularly contrastive learning, have been widely employed to extract action representations from unlabeled data. In this paper, we propose an Ensemble framework for Contrastive Learning of Representation (EnsCLR) to preform unsupervised skeleton-based action recognition. Concretely, Queue Extension method is devised to generate discriminative representation by aggregating the ensemble information from multiple pipelines. Furtherly, Ensemble Nearest Neighbors Mining (ENNM) method is utilized to excavate the most similar samples from the unlabeled data as positive samples, which alleviates the false-negative samples problem caused by the disregard of category label. The experiments with extensive evaluation protocols show that EnsCLR outperforms previous state-of-the-art methods on NTU60, NTU120, and PKU-MMD datasets.
期刊介绍:
The central focus of this journal is the computer analysis of pictorial information. Computer Vision and Image Understanding publishes papers covering all aspects of image analysis from the low-level, iconic processes of early vision to the high-level, symbolic processes of recognition and interpretation. A wide range of topics in the image understanding area is covered, including papers offering insights that differ from predominant views.
Research Areas Include:
• Theory
• Early vision
• Data structures and representations
• Shape
• Range
• Motion
• Matching and recognition
• Architecture and languages
• Vision systems