Annotation of real-world data for artificial intelligence systems : 9th international workshop, ARDUOUS 2025, Bologna Italy, October 25-26, 2025, proceedings. International Workshop on Annotation of Real-World Data for Artificial Intell...最新文献

筛选

英文中文

Towards Practical, Best Practice Video Annotation to Support Human Activity Recognition. 走向实用，最佳实践视频注释支持人类活动识别。

Annotation of real-world data for artificial intelligence systems : 9th international workshop, ARDUOUS 2025, Bologna Italy, October 25-26, 2025, proceedings. International Workshop on Annotation of Real-World Data for Artificial Intell... Pub Date : 2026-01-01 Epub Date: 2025-10-24 DOI: 10.1007/978-3-032-09117-8_6

Hoan Tran, Veronika Potter, Umberto Mazzucchelli, Dinesh John, Stephen Intille

{"title":"Towards Practical, Best Practice Video Annotation to Support Human Activity Recognition.","authors":"Hoan Tran, Veronika Potter, Umberto Mazzucchelli, Dinesh John, Stephen Intille","doi":"10.1007/978-3-032-09117-8_6","DOIUrl":"10.1007/978-3-032-09117-8_6","url":null,"abstract":"Researchers need ground-truth activity annotations to train and evaluate wearable-sensor-based activity recognition models. Often-times, researchers establish ground truth by annotating the video recorded while someone engages in activity wearing sensors. The \"gold-standard\" video annotation practice requires two trained annotators independently annotating the same footage with a third domain expert resolving disagreements. Such annotation is laborious, and so widely-used datasets have often been annotated using only a single annotator per video. Because the research community is moving towards collecting data of more complex behaviors from free-living people 24/7 and annotating more granular, fleeting activities, the annotation task grows even more challenging; the single-annotator approach may yield inaccuracies. We investigated a \"silver-standard\" approach: rather than using two independent annotation passes, a second annotator revises the work of the first annotator. The proposed approach reduced the total annotation time by 33% compared to the gold-standard approach, with near-equivalent annotation quality. The silver-standard label was in higher agreement with the gold-standard label than the single-annotator label, with Cohen's κ of 0.77 and 0.68 respectively on a 16.4 h video. The silver-standard labels also had higher inter-rater reliability than the single-annotator labels, with the respective mean Cohen's κ across six videos (92 h of total footage) of 0.79 and 0.68.","PeriodicalId":521201,"journal":{"name":"Annotation of real-world data for artificial intelligence systems : 9th international workshop, ARDUOUS 2025, Bologna Italy, October 25-26, 2025, proceedings. International Workshop on Annotation of Real-World Data for Artificial Intell...","volume":"2706 ","pages":"94-118"},"PeriodicalIF":0.0,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12799097/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145971670","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0