Jeffrey A Turner, Elaine T Reiche, Matthew T Hartshorne, Connor C Lee, Joanna M Blodgett, Darin A Padua
{"title":"Open Source, Open Science: Development of OpenLESS as the Automated Landing Error Scoring System.","authors":"Jeffrey A Turner, Elaine T Reiche, Matthew T Hartshorne, Connor C Lee, Joanna M Blodgett, Darin A Padua","doi":"10.1101/2024.11.28.24318160","DOIUrl":null,"url":null,"abstract":"<p><strong>Context: </strong>The Open Landing Error Scoring System (OpenLESS) is a novel development aimed at automating the LESS for assessment of lower extremity movement quality during a jump-landing task. With increasing utilization of clinical measures to monitor outcomes and limited time during clinical visits for a lengthy analysis of functional movement, there is a pressing need to extend automation efforts. Addressing these issues, OpenLESS is an open-source tool that utilizes a freely available markerless motion capture system to automate the LESS using three-dimensional kinematics.</p><p><strong>Objective: </strong>To describe the development of OpenLESS, examine the validity against expert rater LESS scores in healthy and clinically relevant cohorts, and assess the intersession reliability collected across four time points in an athlete cohort.</p><p><strong>Design: </strong>Observational.</p><p><strong>Participants: </strong>92 participants (72 females and 20 males, mean age 23.3 years) from healthy, post-anterior cruciate ligament reconstruction (ACLR; median 33 months since surgery), and amateur athlete cohorts.</p><p><strong>Main outcome measures: </strong>A software package, \"OpenLESS,\" was developed to interpret movement quality (LESS score) from kinematics captured from markerless motion capture. Validity and reliability were assessed with intraclass correlation coefficients (ICC), standard error of measure (SEM), and minimal detectable change (MDC).</p><p><strong>Results: </strong>OpenLESS agreed well with expert rater LESS scores for healthy (ICC <sub>2, <i>k</i></sub> =0.79) and clinically relevant, post-ACLR cohorts (ICC <sub>2, <i>k</i></sub> =0.88). The automated OpenLESS system reduced scoring time, processing all 159 trials in under 15 minutes compared to the 18.5 hours (7 minutes per trial) required for manual expert rater scoring. When tested outside laboratory conditions, OpenLESS showed excellent reliability across repeated sessions (ICC <sub>2, <i>k</i></sub> >0.89), with a SEM of 0.98 errors and MDC of 2.72 errors.</p><p><strong>Conclusion: </strong>OpenLESS shows promise as an efficient, automated tool for clinically assessing jump-landing quality, with good validity versus experts in healthy and post-ACLR populations, and excellent field reliability, addressing the need for objective movement analysis.</p><p><strong>Key points: </strong>OpenLESS accurately detected jump-landing events (ICC>0.99) using markerless motion capture, validating its use as an alternative to laboratory-based force plate measurements.The automated scoring system showed good agreement with expert raters in healthy (ICC=0.79) and post-ACLR (ICC=0.88) populations.OpenLESS demonstrated good to excellent test-retest reliability (ICC=0.89) across multiple testing sessions, with minimal score variation, supporting its utility for longitudinal movement assessment.</p>","PeriodicalId":94281,"journal":{"name":"medRxiv : the preprint server for health sciences","volume":" ","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11623740/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"medRxiv : the preprint server for health sciences","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1101/2024.11.28.24318160","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0
Abstract
Context: The Open Landing Error Scoring System (OpenLESS) is a novel development aimed at automating the LESS for assessment of lower extremity movement quality during a jump-landing task. With increasing utilization of clinical measures to monitor outcomes and limited time during clinical visits for a lengthy analysis of functional movement, there is a pressing need to extend automation efforts. Addressing these issues, OpenLESS is an open-source tool that utilizes a freely available markerless motion capture system to automate the LESS using three-dimensional kinematics.
Objective: To describe the development of OpenLESS, examine the validity against expert rater LESS scores in healthy and clinically relevant cohorts, and assess the intersession reliability collected across four time points in an athlete cohort.
Design: Observational.
Participants: 92 participants (72 females and 20 males, mean age 23.3 years) from healthy, post-anterior cruciate ligament reconstruction (ACLR; median 33 months since surgery), and amateur athlete cohorts.
Main outcome measures: A software package, "OpenLESS," was developed to interpret movement quality (LESS score) from kinematics captured from markerless motion capture. Validity and reliability were assessed with intraclass correlation coefficients (ICC), standard error of measure (SEM), and minimal detectable change (MDC).
Results: OpenLESS agreed well with expert rater LESS scores for healthy (ICC 2, k =0.79) and clinically relevant, post-ACLR cohorts (ICC 2, k =0.88). The automated OpenLESS system reduced scoring time, processing all 159 trials in under 15 minutes compared to the 18.5 hours (7 minutes per trial) required for manual expert rater scoring. When tested outside laboratory conditions, OpenLESS showed excellent reliability across repeated sessions (ICC 2, k >0.89), with a SEM of 0.98 errors and MDC of 2.72 errors.
Conclusion: OpenLESS shows promise as an efficient, automated tool for clinically assessing jump-landing quality, with good validity versus experts in healthy and post-ACLR populations, and excellent field reliability, addressing the need for objective movement analysis.
Key points: OpenLESS accurately detected jump-landing events (ICC>0.99) using markerless motion capture, validating its use as an alternative to laboratory-based force plate measurements.The automated scoring system showed good agreement with expert raters in healthy (ICC=0.79) and post-ACLR (ICC=0.88) populations.OpenLESS demonstrated good to excellent test-retest reliability (ICC=0.89) across multiple testing sessions, with minimal score variation, supporting its utility for longitudinal movement assessment.
背景:开放式着陆错误评分系统(OpenLESS)是一项新的发展,旨在自动评估在起跳着陆任务中下肢运动质量的LESS。随着越来越多地利用临床措施来监测结果,以及在临床访问期间对功能运动进行冗长分析的时间有限,迫切需要扩展自动化工作。为了解决这些问题,OpenLESS是一个开源工具,它利用免费可用的无标记运动捕捉系统来使用三维运动学自动化LESS。目的:描述OpenLESS的发展,在健康和临床相关队列中检验专家评分的有效性,并评估在运动员队列中收集的四个时间点的间歇信度。设计:观察。参与者:92名参与者(72名女性和20名男性,平均年龄23.3岁),来自健康的前交叉韧带重建(ACLR;中位术后33个月)和业余运动员队列。主要结果测量:开发了一个软件包“OpenLESS”,用于从无标记运动捕获捕获的运动学中解释运动质量(LESS评分)。用类内相关系数(ICC)、测量标准误差(SEM)和最小可检测变化(MDC)来评估效度和信度。结果:在健康(ICC 2, k =0.79)和临床相关的aclr后队列(ICC 2, k =0.88)中,OpenLESS与专家评分的LESS评分非常一致。自动化的OpenLESS系统减少了评分时间,在15分钟内处理了所有159个试验,而人工专家评分需要18.5小时(每次试验7分钟)。在非实验室条件下测试时,OpenLESS在重复会话中表现出出色的可靠性(ICC 2, k >0.89), SEM误差为0.98,MDC误差为2.72。结论:OpenLESS有望成为一种高效、自动化的临床评估跳跃着落质量的工具,在健康人群和aclr后人群中具有良好的有效性,并且具有出色的现场可靠性,满足了客观运动分析的需求。重点:OpenLESS使用无标记动作捕捉技术准确检测跳跃着陆事件(ICC >.99),验证了其作为实验室力板测量的替代方案。在健康人群(ICC=0.79)和aclr后人群(ICC=0.88)中,自动评分系统与专家评分者表现出良好的一致性。OpenLESS在多个测试会话中表现出良好到卓越的测试-重测试可靠性(ICC=0.89),分数变化最小,支持其纵向运动评估的效用。