Automatic recognition of surgical phase of robot-assisted radical prostatectomy based on artificial intelligence deep-learning model and its application in surgical skill evaluation: a joint study of 18 medical education centers.

Abstract

Background: Surgical proficiency influences surgical quality and patient outcomes in robot-assisted radical prostatectomy (RARP). Manual video evaluations are labor-intensive and lack standardized objective metrics. Herein, we aimed to develop an artificial intelligence (AI) deep-learning model that can identify the surgical phases in RARP videos and create a parameter-based scoring system to distinguish experts from novice surgeons based on the results of the AI model.

Methods: A dataset of 410 RARP videos from 18 Japanese medical institutions was analyzed. The videos were annotated into 11 phases and divided into training and testing sets. Surgeons were categorized as experts or novices based on their RARP experience. We developed a deep-learning-based surgical phase classification model and compared the phase duration, number of transitions between phases, and AI confidence scores (AICS) between the groups based on the model's output. Key parameters were standardized and identified using stepwise multivariate logistic regression. A surgical skill scoring system was constructed based on the receiver operating characteristic curve cut-off values.

Results: Of the 213 videos, 99 were used for training, 20 for validation, and 94 for testing (61 experts and 33 novices). The model achieved an accuracy of 0.89 in identifying surgical phases. The experts had significantly shorter durations in phases 2-8 and higher AICS than the novices. Stepwise analysis identified phases 2 (Retzius space expansion), 7 (dorsal venous complex incision, apex treatment, hemostasis), and 8 (urethrovesical anastomosis) and the AICS as key predictors of expertise. The scoring system developed from these variables effectively distinguished experts from novices with an accuracy of 86.2%.

Conclusions: The developed AI model revealed that the duration of several surgical phases and AICS are key parameters in assessing surgical skill proficiency in RARP. The new scoring system established based on these indicators reliably differentiates expert from novice surgeons.