The Predictive Value of Basic Laparoscopic Evaluations on Complex Cases Using the Society for Improving Medical Procedural Learning Database

Abstract

Introduction

The transference of technical skills from basic to complex procedures is assumed–but untested–during training and board certification. We examine whether resident performance on frequently performed, basic laparoscopic procedures can predict performance on their first complex laparoscopic procedure.

Methods

This cross-sectional, retrospective study employs the Society for Improving Medical Procedural Learning collaborative operating room database from 2016 to 2023 using attending-level evaluations of general surgery residents at accredited teaching hospitals. Using cumulative instances of prior basic laparoscopic procedures, we applied five machine learning algorithms to predict the performance on the resident's first complex procedure. Procedures were labeled “basic” or “complex” according to Accreditation Council on Graduate Medical Education definitions.

Results

7792 attending evaluations of 899 unique residents were included. The number of residents with evaluations of prior basic procedures varied and dropped linearly from 646 (2 prior) to 180 (10 prior). The best performing random forest model obtained a mean squared error of 0.23 (95% confidence interval 0.17-0.29) using 3 prior procedures. Resident postgraduate year was the most important predictor, and simple univariate regression on this variable yielded a mean squared error of 0.33 (95% confidence interval 0.31-0.37).

Conclusions

Basic laparoscopic performance ratings were moderately effective in predicting subsequent performance on complex procedures but offered little additional predictive value over postgraduate year alone.