Performance Evaluation of AI-Powered Pelvic Lymph Nodes Dissection Support System

Study Objective

The objective was to build a pelvic lymph node dissection support system using AI, evaluate the performance of the model, and verify whether this model provides an additional effect on physician organ recognition ability.

Design

This is a retrospective cohort study.

Setting

Using image data from 263 cases of pelvic lymphadenectomy from a national multi-center surgical database (111 gynecology, 118 colorectal, 34 urology), totaling 19,301 images, we constructed four organ recognition models (ureter, obturator nerve, external iliac artery/vein) using Feature Pyramid Networks (FPN). Subsequently, total of 1,920 videos were then created, including videos with and without each organ present.

Patients or Participants

Four obstetricians and gynecologists, two colorectal surgeons, two urologists.

Interventions

In the performance evaluation test, the accuracy of each organ was measured as Dice coefficient. In the additional evaluation test, surgeons were tested to determine the presence or absence of the organs and their locations in the videos without AI support. Next, the same test was conducted using videos with AI support.

Measurements and Main Results

In the performance evaluation test, the Dice coefficients were: ureter 0.700, nerve 0.835, artery 0.864, vein 0.862. In the additional effect test, sensitivity increased significantly for all organs except the artery: ureter +20.0% (43.4% → 63.4%), nerve +7.2% (68.4% → 75.6%), artery +5.9% (69.7% → 75.6%), and vein +11.5% (69.1% → 80.6%). Specificity also improved: ureter +4.4% (86.9% → 91.3%), nerve +7.5% (85.3% → 92.8%), artery +1.9% (93.4% → 95.3%), and vein +7.9% (83.4% → 91.3%), with no decline due to AI support.

Conclusion

The AI model showed a notable enhancement in surgeons' organ recognition ability. Future tests will involve surgeons of varying skill levels across three specialties to validate the model.