Development and validation of an objective assessment of surgical skill in arthroscopic management of meniscal tear: A pilot study

Introduction

As resident evaluation moves to a competency-based system, validated tools for assessment of surgical skill are increasingly important. We created and validated a checklist to measure resident surgical skill for arthroscopic management of meniscal tear.

Materials and Methods

Using a Delphi survey method, we created an objective, structured assessment of surgical skill for treatment of meniscal tears. The Meniscus Treatment Task List (MTTL) comprises 5 domains: diagnostic arthroscopy, medial meniscectomy, lateral meniscectomy, medial meniscal repair, and lateral meniscal repair. Orthopaedic surgery residents were recruited to perform diagnostic arthroscopy, partial meniscectomies, and all-inside meniscal repairs with cadaveric models. Arthroscopic videos were graded by fellowship-trained surgeons using the MTTL and the validated Arthroscopic Surgical Skill Evaluation Tool (ASSET) global rating scale (GRS). Postgraduate year (PGY), operative time, and case logs were recorded for each resident. Data were analysed using bivariate correlation, analysis of variance, pairwise comparison, Pearson's correlation coefficient, and intraclass correlation coefficient. α=0.05.

Results

Twenty-two orthopaedic surgery residents (PGY1–PGY4) participated. MTTL scores were higher in the PGY4 class than in the PGY1 class (mean difference, 11 points, p = 0.04). Operative time was inversely correlated with number of cases logged (r = –0.53, p = 0.01), number of arthroscopic cases logged (r = –0.50, p = 0.02), and MTTL score (r = –0.46, p = 0.03). MTTL score was positively correlated with number of cases (r = 0.44, p = 0.04) and number of arthroscopic cases logged (r = 0.50, p = 0.02). MTTL scores were positively correlated with the ASSET GRS (r = 0.71, p<0.001). Intraclass correlation coefficient of 0.89 and Pearson's correlation coefficient of 0.89 demonstrated strong interrater reliability of MTTL scores (p<0.01).

Conclusions

This pilot study demonstrates the validity and reliability of the MTTL for assessing resident proficiency in arthroscopic management of meniscal tears in cadaveric specimens. Expansion of this model to other orthopaedic procedures for objective assessment of surgical skill may be useful.