Adhitya Ramamurthi MD, MS , Bhabishya Neupane BS , Priya Deshpande PhD , Ryan Hanson MS , Kellie R. Brown MD , Kathleen K. Christians MD , Douglas B. Evans MD , Anai N. Kothari MD, MS
{"title":"Development and validation of an artificial intelligence system for surgical case length prediction","authors":"Adhitya Ramamurthi MD, MS , Bhabishya Neupane BS , Priya Deshpande PhD , Ryan Hanson MS , Kellie R. Brown MD , Kathleen K. Christians MD , Douglas B. Evans MD , Anai N. Kothari MD, MS","doi":"10.1016/j.surg.2024.09.051","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>Accurate case length estimation is a vital part of optimizing operating room use; however, significant inaccuracies exist with current solutions. The purpose of this study was to develop and validate an artificial intelligence system for improved surgical case length prediction by applying natural language processing and machine-learning methods.</div></div><div><h3>Methods</h3><div>All inpatient elective surgical cases longer than 30 minutes completed between 2017 and 2023 at a single, quaternary care hospital were considered. Data were split into training, test, and hold-out validation for model training and testing. Linear regression, CategoricalBoost, and feed-forward neural network each were trained and used embeddings created by bidirectional encoder representations from transformers or a bidirectional encoder representations from transformers model pretrained on clinical text. The average root mean squared error and mean absolute error were calculated for each model.</div></div><div><h3>Results</h3><div>A total of 125,493 cases were included. The highest performing model was the CategoricalBoost Regressor with bidirectional encoder representations from transformers model pretrained on clinical text embeddings (mean absolute error, 46.4 minutes), which was lower than the existing electronic health record estimates (120.0 minutes, <em>P</em> < 0.001). Accurate estimation of case length was defined as within ±20% of the actual case length with our model having 48% accuracy vs 17% accuracy for electronic health record–generated estimates.</div></div><div><h3>Conclusion</h3><div>An artificial intelligence model for surgical case length estimation outperforms existing institutional electronic health record predictions. On average, the estimate improved by 62% and approximately 2.8× the number of cases were correctly estimated. This study shows the successful development of machine learning models using advanced natural language processing techniques for improved surgical case length prediction.</div></div>","PeriodicalId":22152,"journal":{"name":"Surgery","volume":"179 ","pages":"Article 108942"},"PeriodicalIF":3.2000,"publicationDate":"2024-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Surgery","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S003960602400919X","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"SURGERY","Score":null,"Total":0}
引用次数: 0
Abstract
Background
Accurate case length estimation is a vital part of optimizing operating room use; however, significant inaccuracies exist with current solutions. The purpose of this study was to develop and validate an artificial intelligence system for improved surgical case length prediction by applying natural language processing and machine-learning methods.
Methods
All inpatient elective surgical cases longer than 30 minutes completed between 2017 and 2023 at a single, quaternary care hospital were considered. Data were split into training, test, and hold-out validation for model training and testing. Linear regression, CategoricalBoost, and feed-forward neural network each were trained and used embeddings created by bidirectional encoder representations from transformers or a bidirectional encoder representations from transformers model pretrained on clinical text. The average root mean squared error and mean absolute error were calculated for each model.
Results
A total of 125,493 cases were included. The highest performing model was the CategoricalBoost Regressor with bidirectional encoder representations from transformers model pretrained on clinical text embeddings (mean absolute error, 46.4 minutes), which was lower than the existing electronic health record estimates (120.0 minutes, P < 0.001). Accurate estimation of case length was defined as within ±20% of the actual case length with our model having 48% accuracy vs 17% accuracy for electronic health record–generated estimates.
Conclusion
An artificial intelligence model for surgical case length estimation outperforms existing institutional electronic health record predictions. On average, the estimate improved by 62% and approximately 2.8× the number of cases were correctly estimated. This study shows the successful development of machine learning models using advanced natural language processing techniques for improved surgical case length prediction.
期刊介绍:
For 66 years, Surgery has published practical, authoritative information about procedures, clinical advances, and major trends shaping general surgery. Each issue features original scientific contributions and clinical reports. Peer-reviewed articles cover topics in oncology, trauma, gastrointestinal, vascular, and transplantation surgery. The journal also publishes papers from the meetings of its sponsoring societies, the Society of University Surgeons, the Central Surgical Association, and the American Association of Endocrine Surgeons.