Rikke Groth Olsen, Flemming Bjerrum, Annarita Ghosh Andersen, Lars Konge, Andreas Røder, Morten Bo Søndergaard Svendsen
{"title":"Development of an artificial intelligence algorithm for automated surgical gestures annotation.","authors":"Rikke Groth Olsen, Flemming Bjerrum, Annarita Ghosh Andersen, Lars Konge, Andreas Røder, Morten Bo Søndergaard Svendsen","doi":"10.1007/s11701-025-02556-2","DOIUrl":null,"url":null,"abstract":"<p><p>Surgical gestures analysis is a promising method to assess surgical procedure quality, but manual annotation is time-consuming. We aimed to develop a recurrent neural network for automated surgical gesture annotation using simulated robot-assisted radical prostatectomies. We have previously manually annotated 161 videos with five different surgical gestures (Regular dissection, Hemostatic control, Clip application, Needle handling, and Suturing). We created a model consisting of two neural networks: a pre-trained feature extractor (VisionTransformer using Imagenet) and a classification head (recurrent neural network with a Long Short-Term Memory (LSTM(128) and fully connected layer)). The data set was split into a training + validation set and a test set. The trained model labeled input sequences with one of the five surgical gestures. The overall performance of the neural networks was assessed by metrics for multi-label classification and defined Total Agreement, an extended version of Intersection over Union (IoU). Our neural network could predict the class of surgical gestures with an Area Under the Curve (AUC) of 0.95 (95% CI 0.93-0.96) and an F1-score of 0.71 (95% CI 0.67-0.75). The network could classify each surgical gesture with high accuracies (0.84-0.97) and high specificities (0.90-0.99), but with lower sensitivities (0.62-0.81). The average Total Agreement for each gesture class was between 0.72 (95% CI ± 0.03) and 0.91 (95% CI ± 0.02). We successfully developed a high-performing neural network to analyze gestures in simulated surgical procedures. Our next step is to use the network to annotate videos and evaluate their efficacy in predicting patient outcomes.</p>","PeriodicalId":47616,"journal":{"name":"Journal of Robotic Surgery","volume":"19 1","pages":"404"},"PeriodicalIF":2.2000,"publicationDate":"2025-07-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12274238/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Robotic Surgery","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s11701-025-02556-2","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"SURGERY","Score":null,"Total":0}
引用次数: 0
Abstract
Surgical gestures analysis is a promising method to assess surgical procedure quality, but manual annotation is time-consuming. We aimed to develop a recurrent neural network for automated surgical gesture annotation using simulated robot-assisted radical prostatectomies. We have previously manually annotated 161 videos with five different surgical gestures (Regular dissection, Hemostatic control, Clip application, Needle handling, and Suturing). We created a model consisting of two neural networks: a pre-trained feature extractor (VisionTransformer using Imagenet) and a classification head (recurrent neural network with a Long Short-Term Memory (LSTM(128) and fully connected layer)). The data set was split into a training + validation set and a test set. The trained model labeled input sequences with one of the five surgical gestures. The overall performance of the neural networks was assessed by metrics for multi-label classification and defined Total Agreement, an extended version of Intersection over Union (IoU). Our neural network could predict the class of surgical gestures with an Area Under the Curve (AUC) of 0.95 (95% CI 0.93-0.96) and an F1-score of 0.71 (95% CI 0.67-0.75). The network could classify each surgical gesture with high accuracies (0.84-0.97) and high specificities (0.90-0.99), but with lower sensitivities (0.62-0.81). The average Total Agreement for each gesture class was between 0.72 (95% CI ± 0.03) and 0.91 (95% CI ± 0.02). We successfully developed a high-performing neural network to analyze gestures in simulated surgical procedures. Our next step is to use the network to annotate videos and evaluate their efficacy in predicting patient outcomes.
期刊介绍:
The aim of the Journal of Robotic Surgery is to become the leading worldwide journal for publication of articles related to robotic surgery, encompassing surgical simulation and integrated imaging techniques. The journal provides a centralized, focused resource for physicians wishing to publish their experience or those wishing to avail themselves of the most up-to-date findings.The journal reports on advance in a wide range of surgical specialties including adult and pediatric urology, general surgery, cardiac surgery, gynecology, ENT, orthopedics and neurosurgery.The use of robotics in surgery is broad-based and will undoubtedly expand over the next decade as new technical innovations and techniques increase the applicability of its use. The journal intends to capture this trend as it develops.