{"title":"学生竞赛(科技创新) ID 1970388","authors":"Mehdy Dousty, David J. Fleet, J. Zariffa","doi":"10.46292/sci23-1970388s","DOIUrl":null,"url":null,"abstract":"The evaluation of hand function after spinal cord injury (SCI) is conducted in clinical settings, which may not accurately reflect hand function in the real world, thereby limiting the efficacy assessment of new treatments. Wearable cameras, also known as egocentric video, are a novel method to evaluate hand function in non-clinical environments. Nonetheless, manual processing of vast quantities of complex video data is difficult, highlighting the need for automated data analysis. The objective of this study was to automatically identify distinct hand postures in egocentric video using unsupervised machine learning. Seventeen participants with cervical SCI recorded activities of daily living in a home simulation laboratory. A hand pose estimation algorithm was applied on detected hands to determine 2D joint locations, which were lifted to 3D coordinates. The resulting hand posture information was subjected to a number of clustering techniques. Hand grasps were manually labelled into four categories for evaluation purposes: power, precision, intermediate, and non-prehensile. K-Means clustering consistently exhibited the highest Silhouette score, which reflects the presence of discrete clusters in the data. When comparing with manual annotations, Spectral Clustering applied to a feature space consisting of 2D pose estimation with confidence scores yield the best performance as quantified by maximum match (0.48), Fowlkes-Mallows score (0.46), and normalized mutual information (0.22). This is the first attempt to develop an unsupervised, data-driven hand taxonomy for individuals with SCI using wearable technology. The findings suggest that the method is capable of grouping similar hand grasps.","PeriodicalId":46769,"journal":{"name":"Topics in Spinal Cord Injury Rehabilitation","volume":"89 1","pages":""},"PeriodicalIF":2.4000,"publicationDate":"2023-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Student Competition (Technology Innovation) ID 1970388\",\"authors\":\"Mehdy Dousty, David J. Fleet, J. Zariffa\",\"doi\":\"10.46292/sci23-1970388s\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The evaluation of hand function after spinal cord injury (SCI) is conducted in clinical settings, which may not accurately reflect hand function in the real world, thereby limiting the efficacy assessment of new treatments. Wearable cameras, also known as egocentric video, are a novel method to evaluate hand function in non-clinical environments. Nonetheless, manual processing of vast quantities of complex video data is difficult, highlighting the need for automated data analysis. The objective of this study was to automatically identify distinct hand postures in egocentric video using unsupervised machine learning. Seventeen participants with cervical SCI recorded activities of daily living in a home simulation laboratory. A hand pose estimation algorithm was applied on detected hands to determine 2D joint locations, which were lifted to 3D coordinates. The resulting hand posture information was subjected to a number of clustering techniques. Hand grasps were manually labelled into four categories for evaluation purposes: power, precision, intermediate, and non-prehensile. K-Means clustering consistently exhibited the highest Silhouette score, which reflects the presence of discrete clusters in the data. When comparing with manual annotations, Spectral Clustering applied to a feature space consisting of 2D pose estimation with confidence scores yield the best performance as quantified by maximum match (0.48), Fowlkes-Mallows score (0.46), and normalized mutual information (0.22). This is the first attempt to develop an unsupervised, data-driven hand taxonomy for individuals with SCI using wearable technology. The findings suggest that the method is capable of grouping similar hand grasps.\",\"PeriodicalId\":46769,\"journal\":{\"name\":\"Topics in Spinal Cord Injury Rehabilitation\",\"volume\":\"89 1\",\"pages\":\"\"},\"PeriodicalIF\":2.4000,\"publicationDate\":\"2023-09-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Topics in Spinal Cord Injury Rehabilitation\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.46292/sci23-1970388s\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"REHABILITATION\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Topics in Spinal Cord Injury Rehabilitation","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.46292/sci23-1970388s","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"REHABILITATION","Score":null,"Total":0}
Student Competition (Technology Innovation) ID 1970388
The evaluation of hand function after spinal cord injury (SCI) is conducted in clinical settings, which may not accurately reflect hand function in the real world, thereby limiting the efficacy assessment of new treatments. Wearable cameras, also known as egocentric video, are a novel method to evaluate hand function in non-clinical environments. Nonetheless, manual processing of vast quantities of complex video data is difficult, highlighting the need for automated data analysis. The objective of this study was to automatically identify distinct hand postures in egocentric video using unsupervised machine learning. Seventeen participants with cervical SCI recorded activities of daily living in a home simulation laboratory. A hand pose estimation algorithm was applied on detected hands to determine 2D joint locations, which were lifted to 3D coordinates. The resulting hand posture information was subjected to a number of clustering techniques. Hand grasps were manually labelled into four categories for evaluation purposes: power, precision, intermediate, and non-prehensile. K-Means clustering consistently exhibited the highest Silhouette score, which reflects the presence of discrete clusters in the data. When comparing with manual annotations, Spectral Clustering applied to a feature space consisting of 2D pose estimation with confidence scores yield the best performance as quantified by maximum match (0.48), Fowlkes-Mallows score (0.46), and normalized mutual information (0.22). This is the first attempt to develop an unsupervised, data-driven hand taxonomy for individuals with SCI using wearable technology. The findings suggest that the method is capable of grouping similar hand grasps.
期刊介绍:
Now in our 22nd year as the leading interdisciplinary journal of SCI rehabilitation techniques and care. TSCIR is peer-reviewed, practical, and features one key topic per issue. Published topics include: mobility, sexuality, genitourinary, functional assessment, skin care, psychosocial, high tetraplegia, physical activity, pediatric, FES, sci/tbi, electronic medicine, orthotics, secondary conditions, research, aging, legal issues, women & sci, pain, environmental effects, life care planning