Does the ArthroSim™ Shoulder Simulation Discriminate Between the Novice and Expert User? An External Validation Study of the ArthroSim™ Shoulder Arthroscopy Simulator.

Abstract

Introduction: Our aim is to investigate the face validity and the construct validity of the ArthroSim™ virtual reality shoulder arthroscopy simulator (Touch of Life Technologies, Inc., Aurora, Colorado).

Materials and methods: Fifteen doctors participated, with five from each of the following levels: novice with no arthroscopies performed, intermediate with less than 50 arthroscopies performed, and expert with over 100 arthroscopies performed. They first undertook a simulation task on the ArthroSim™ simulator, with the simulation and their body movements recorded. Each participant then completed a Likert scale questionnaire after their simulation task evaluating the face validity of the machine. The video recordings were rated by two shoulder surgeons independently using the Global Rating Scale for Shoulder Arthroplasty (GRSSA), Arthroscopic Surgical Skill Evaluation Tool (ASSET), and Imperial Global Arthroscopy Rating Scale (IGARS) scoring systems for construct validation.

Results: All three scales demonstrated high internal consistency GRSSA (0.976 ), ASSET (0.980), and IGARS (0.979). The intraclass correlation coefficient demonstrated high agreement between the assessors: GRSSA (0.88), ASSET (0.90), and IGARS (0.89). Construct validity was evaluated using Kruskal-Wallis one-way analysis of variance: GRSSA (chi square test: 19.828; p<0.001); ASSET (chi square test: 19.62; p<0.001); and IGARS (chi square test: 19.63; p<0.001). This demonstrates that the three scoring systems distinguish significantly between participants with different levels of experience utilizing a virtual reality simulator.

Conclusion: Our study has shown that the ArthoSim™ simulator can accurately distinguish between experience level of the participants. It is more effective in reproducing a training environment as reality for less experienced surgeons. All participants agreed that it is valuable in developing surgical training for the novice and intermediate skill level.