Hermann Anetzberger, Andreas Kugler, Michael Mohr, Florian Haasters, Stephan Reppenhagen, Roland Becker
{"title":"首次验证了使用诊断性关节镜技能评分（DASS 2.0）的自动评分系统，用于评估虚拟现实关节镜的熟练程度。","authors":"Hermann Anetzberger, Andreas Kugler, Michael Mohr, Florian Haasters, Stephan Reppenhagen, Roland Becker","doi":"10.1002/ksa.12670","DOIUrl":null,"url":null,"abstract":"<p><strong>Purpose: </strong>Proficiency-based simulator training is a promising approach for learning the complex motor skills required for arthroscopy. However, its implementation requires an objective assessment tool to assess residents' arthroscopic skills. To address this need, an automated diagnostic arthroscopy skill score (DASS) was developed and validated as a replacement for manual scoring.</p><p><strong>Methods: </strong>An automated measurement system of arthroscopic skills was developed, replacing the manual assessment parameters of the DASS with objective measurement criteria. Data from arthroscopies performed by 20 experts were used to establish threshold values for scoring. To validate the new method, 125 videos of residents were recorded and evaluated by five raters. The results of the automated evaluation were then compared with those of established manual assessment. To assess the reliability of the manual evaluation, the intraclass correlation coefficient (ICC) and minimum detectable change (MDC) were calculated. Methodological agreement was evaluated using linear least squares regression and the Bland-Altman method.</p><p><strong>Results: </strong>A good to excellent level of reliability was found among the five raters (ICC for DASS<sub>part1</sub> = 0.89, 95% confidence interval [CI] = 0.84-0.92). The calculated MDC was 5.0 points. High methodological agreement was found between the manual and automated evaluations of the DASS. The 95% CI for the slope of the regression line included 1, and the 95% CI for the intercept included 0. According to the Bland-Altman method, the mean difference between manual and automated evaluations was 4.1 ± 5.4 points, and the scattering of the measurement differences was uniformly distributed, regardless of the total score.</p><p><strong>Conclusions: </strong>Automated DASS measurement is a valid and reliable tool for assessing arthroscopic skills. Its advantages, such as rater independence, precise and objective measurements, and immediate evaluation, make it a powerful tool for evaluating arthroscopic performance during simulation training.</p><p><strong>Level of evidence: </strong>Level II.</p>","PeriodicalId":17880,"journal":{"name":"Knee Surgery, Sports Traumatology, Arthroscopy","volume":" ","pages":""},"PeriodicalIF":3.3000,"publicationDate":"2025-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"First validated automated scoring system using the diagnostic arthroscopy skill score (DASS 2.0) for assessing proficiency in virtual reality arthroscopy.\",\"authors\":\"Hermann Anetzberger, Andreas Kugler, Michael Mohr, Florian Haasters, Stephan Reppenhagen, Roland Becker\",\"doi\":\"10.1002/ksa.12670\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Purpose: </strong>Proficiency-based simulator training is a promising approach for learning the complex motor skills required for arthroscopy. However, its implementation requires an objective assessment tool to assess residents' arthroscopic skills. To address this need, an automated diagnostic arthroscopy skill score (DASS) was developed and validated as a replacement for manual scoring.</p><p><strong>Methods: </strong>An automated measurement system of arthroscopic skills was developed, replacing the manual assessment parameters of the DASS with objective measurement criteria. Data from arthroscopies performed by 20 experts were used to establish threshold values for scoring. To validate the new method, 125 videos of residents were recorded and evaluated by five raters. The results of the automated evaluation were then compared with those of established manual assessment. To assess the reliability of the manual evaluation, the intraclass correlation coefficient (ICC) and minimum detectable change (MDC) were calculated. Methodological agreement was evaluated using linear least squares regression and the Bland-Altman method.</p><p><strong>Results: </strong>A good to excellent level of reliability was found among the five raters (ICC for DASS<sub>part1</sub> = 0.89, 95% confidence interval [CI] = 0.84-0.92). The calculated MDC was 5.0 points. High methodological agreement was found between the manual and automated evaluations of the DASS. The 95% CI for the slope of the regression line included 1, and the 95% CI for the intercept included 0. According to the Bland-Altman method, the mean difference between manual and automated evaluations was 4.1 ± 5.4 points, and the scattering of the measurement differences was uniformly distributed, regardless of the total score.</p><p><strong>Conclusions: </strong>Automated DASS measurement is a valid and reliable tool for assessing arthroscopic skills. Its advantages, such as rater independence, precise and objective measurements, and immediate evaluation, make it a powerful tool for evaluating arthroscopic performance during simulation training.</p><p><strong>Level of evidence: </strong>Level II.</p>\",\"PeriodicalId\":17880,\"journal\":{\"name\":\"Knee Surgery, Sports Traumatology, Arthroscopy\",\"volume\":\" \",\"pages\":\"\"},\"PeriodicalIF\":3.3000,\"publicationDate\":\"2025-04-08\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Knee Surgery, Sports Traumatology, Arthroscopy\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1002/ksa.12670\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ORTHOPEDICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Knee Surgery, Sports Traumatology, Arthroscopy","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1002/ksa.12670","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ORTHOPEDICS","Score":null,"Total":0}
First validated automated scoring system using the diagnostic arthroscopy skill score (DASS 2.0) for assessing proficiency in virtual reality arthroscopy.
Purpose: Proficiency-based simulator training is a promising approach for learning the complex motor skills required for arthroscopy. However, its implementation requires an objective assessment tool to assess residents' arthroscopic skills. To address this need, an automated diagnostic arthroscopy skill score (DASS) was developed and validated as a replacement for manual scoring.
Methods: An automated measurement system of arthroscopic skills was developed, replacing the manual assessment parameters of the DASS with objective measurement criteria. Data from arthroscopies performed by 20 experts were used to establish threshold values for scoring. To validate the new method, 125 videos of residents were recorded and evaluated by five raters. The results of the automated evaluation were then compared with those of established manual assessment. To assess the reliability of the manual evaluation, the intraclass correlation coefficient (ICC) and minimum detectable change (MDC) were calculated. Methodological agreement was evaluated using linear least squares regression and the Bland-Altman method.
Results: A good to excellent level of reliability was found among the five raters (ICC for DASSpart1 = 0.89, 95% confidence interval [CI] = 0.84-0.92). The calculated MDC was 5.0 points. High methodological agreement was found between the manual and automated evaluations of the DASS. The 95% CI for the slope of the regression line included 1, and the 95% CI for the intercept included 0. According to the Bland-Altman method, the mean difference between manual and automated evaluations was 4.1 ± 5.4 points, and the scattering of the measurement differences was uniformly distributed, regardless of the total score.
Conclusions: Automated DASS measurement is a valid and reliable tool for assessing arthroscopic skills. Its advantages, such as rater independence, precise and objective measurements, and immediate evaluation, make it a powerful tool for evaluating arthroscopic performance during simulation training.
期刊介绍：
Few other areas of orthopedic surgery and traumatology have undergone such a dramatic evolution in the last 10 years as knee surgery, arthroscopy and sports traumatology. Ranked among the top 33% of journals in both Orthopedics and Sports Sciences, the goal of this European journal is to publish papers about innovative knee surgery, sports trauma surgery and arthroscopy. Each issue features a series of peer-reviewed articles that deal with diagnosis and management and with basic research. Each issue also contains at least one review article about an important clinical problem. Case presentations or short notes about technical innovations are also accepted for publication.
The articles cover all aspects of knee surgery and all types of sports trauma; in addition, epidemiology, diagnosis, treatment and prevention, and all types of arthroscopy (not only the knee but also the shoulder, elbow, wrist, hip, ankle, etc.) are addressed. Articles on new diagnostic techniques such as MRI and ultrasound and high-quality articles about the biomechanics of joints, muscles and tendons are included. Although this is largely a clinical journal, it is also open to basic research with clinical relevance.
Because the journal is supported by a distinguished European Editorial Board, assisted by an international Advisory Board, you can be assured that the journal maintains the highest standards.
Official Clinical Journal of the European Society of Sports Traumatology, Knee Surgery and Arthroscopy (ESSKA).
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
