{"title":"SPKDB-Net:一种基于显著部分姿态关键点的重复动作计数双分支网络","authors":"Jinying Wu , Jun Li , Qiming Li","doi":"10.1016/j.cviu.2025.104434","DOIUrl":null,"url":null,"abstract":"<div><div>With the continuous development of deep learning, the field of repetitive action counting is gradually gaining notice from many researchers. Extraction of pose keypoints using human pose estimation networks is proven to be an effective pose-level method. However, the existing pose-level methods have some drawbacks, for example, ignoring the fact that occlusion and unfavourable viewing angles in videos lead to affect the accuracy of pose keypoints extraction. To overcome these problems, we propose a simple but efficient Salient-Part Pose Keypoints-Based Dual-Branch Network (SPKDB-Net). Specifically, we design a dual-branch input channel consisting of a global-based and a salient-part input branch. The global-based input branch is used to input the pose keypoints of the whole body extracted by the human pose estimation network, and the salient-part input branch is used to input the salient-part pose keypoints (<em>i.e.</em>, head, shoulders, and hands). The second branch acts as an auxiliary to the first branch, thus effectively addressing the influence of external factors. In addition, we propose a DFEPM-Module that obtains long-distance dependency between pose keypoints through the attention mechanism, and obtains salient local features fused by the attention mechanism through convolution. Eventually, extensive experiments on the challenging RepCount-pose, UCFRep-pose and Countix-Fitness-pose benchmarks show that our proposed SPKDB-Net achieves state-of-the-art performance.</div></div>","PeriodicalId":50633,"journal":{"name":"Computer Vision and Image Understanding","volume":"259 ","pages":"Article 104434"},"PeriodicalIF":3.5000,"publicationDate":"2025-06-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"SPKDB-Net: A Salient-Part Pose Keypoints-Based Dual-Branch Network for repetitive action counting\",\"authors\":\"Jinying Wu , Jun Li , Qiming Li\",\"doi\":\"10.1016/j.cviu.2025.104434\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>With the continuous development of deep learning, the field of repetitive action counting is gradually gaining notice from many researchers. Extraction of pose keypoints using human pose estimation networks is proven to be an effective pose-level method. However, the existing pose-level methods have some drawbacks, for example, ignoring the fact that occlusion and unfavourable viewing angles in videos lead to affect the accuracy of pose keypoints extraction. To overcome these problems, we propose a simple but efficient Salient-Part Pose Keypoints-Based Dual-Branch Network (SPKDB-Net). Specifically, we design a dual-branch input channel consisting of a global-based and a salient-part input branch. The global-based input branch is used to input the pose keypoints of the whole body extracted by the human pose estimation network, and the salient-part input branch is used to input the salient-part pose keypoints (<em>i.e.</em>, head, shoulders, and hands). The second branch acts as an auxiliary to the first branch, thus effectively addressing the influence of external factors. In addition, we propose a DFEPM-Module that obtains long-distance dependency between pose keypoints through the attention mechanism, and obtains salient local features fused by the attention mechanism through convolution. Eventually, extensive experiments on the challenging RepCount-pose, UCFRep-pose and Countix-Fitness-pose benchmarks show that our proposed SPKDB-Net achieves state-of-the-art performance.</div></div>\",\"PeriodicalId\":50633,\"journal\":{\"name\":\"Computer Vision and Image Understanding\",\"volume\":\"259 \",\"pages\":\"Article 104434\"},\"PeriodicalIF\":3.5000,\"publicationDate\":\"2025-06-30\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Computer Vision and Image Understanding\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1077314225001572\",\"RegionNum\":3,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Computer Vision and Image Understanding","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1077314225001572","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
SPKDB-Net: A Salient-Part Pose Keypoints-Based Dual-Branch Network for repetitive action counting
With the continuous development of deep learning, the field of repetitive action counting is gradually gaining notice from many researchers. Extraction of pose keypoints using human pose estimation networks is proven to be an effective pose-level method. However, the existing pose-level methods have some drawbacks, for example, ignoring the fact that occlusion and unfavourable viewing angles in videos lead to affect the accuracy of pose keypoints extraction. To overcome these problems, we propose a simple but efficient Salient-Part Pose Keypoints-Based Dual-Branch Network (SPKDB-Net). Specifically, we design a dual-branch input channel consisting of a global-based and a salient-part input branch. The global-based input branch is used to input the pose keypoints of the whole body extracted by the human pose estimation network, and the salient-part input branch is used to input the salient-part pose keypoints (i.e., head, shoulders, and hands). The second branch acts as an auxiliary to the first branch, thus effectively addressing the influence of external factors. In addition, we propose a DFEPM-Module that obtains long-distance dependency between pose keypoints through the attention mechanism, and obtains salient local features fused by the attention mechanism through convolution. Eventually, extensive experiments on the challenging RepCount-pose, UCFRep-pose and Countix-Fitness-pose benchmarks show that our proposed SPKDB-Net achieves state-of-the-art performance.
期刊介绍:
The central focus of this journal is the computer analysis of pictorial information. Computer Vision and Image Understanding publishes papers covering all aspects of image analysis from the low-level, iconic processes of early vision to the high-level, symbolic processes of recognition and interpretation. A wide range of topics in the image understanding area is covered, including papers offering insights that differ from predominant views.
Research Areas Include:
• Theory
• Early vision
• Data structures and representations
• Shape
• Range
• Motion
• Matching and recognition
• Architecture and languages
• Vision systems