Development and validation of an artificial intelligence system for surgical case length prediction

IF 3.2 2区 医学 Q1 SURGERY
Surgery Pub Date : 2024-11-29 DOI:10.1016/j.surg.2024.09.051
Adhitya Ramamurthi MD, MS , Bhabishya Neupane BS , Priya Deshpande PhD , Ryan Hanson MS , Kellie R. Brown MD , Kathleen K. Christians MD , Douglas B. Evans MD , Anai N. Kothari MD, MS
{"title":"Development and validation of an artificial intelligence system for surgical case length prediction","authors":"Adhitya Ramamurthi MD, MS ,&nbsp;Bhabishya Neupane BS ,&nbsp;Priya Deshpande PhD ,&nbsp;Ryan Hanson MS ,&nbsp;Kellie R. Brown MD ,&nbsp;Kathleen K. Christians MD ,&nbsp;Douglas B. Evans MD ,&nbsp;Anai N. Kothari MD, MS","doi":"10.1016/j.surg.2024.09.051","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>Accurate case length estimation is a vital part of optimizing operating room use; however, significant inaccuracies exist with current solutions. The purpose of this study was to develop and validate an artificial intelligence system for improved surgical case length prediction by applying natural language processing and machine-learning methods.</div></div><div><h3>Methods</h3><div>All inpatient elective surgical cases longer than 30 minutes completed between 2017 and 2023 at a single, quaternary care hospital were considered. Data were split into training, test, and hold-out validation for model training and testing. Linear regression, CategoricalBoost, and feed-forward neural network each were trained and used embeddings created by bidirectional encoder representations from transformers or a bidirectional encoder representations from transformers model pretrained on clinical text. The average root mean squared error and mean absolute error were calculated for each model.</div></div><div><h3>Results</h3><div>A total of 125,493 cases were included. The highest performing model was the CategoricalBoost Regressor with bidirectional encoder representations from transformers model pretrained on clinical text embeddings (mean absolute error, 46.4 minutes), which was lower than the existing electronic health record estimates (120.0 minutes, <em>P</em> &lt; 0.001). Accurate estimation of case length was defined as within ±20% of the actual case length with our model having 48% accuracy vs 17% accuracy for electronic health record–generated estimates.</div></div><div><h3>Conclusion</h3><div>An artificial intelligence model for surgical case length estimation outperforms existing institutional electronic health record predictions. On average, the estimate improved by 62% and approximately 2.8× the number of cases were correctly estimated. This study shows the successful development of machine learning models using advanced natural language processing techniques for improved surgical case length prediction.</div></div>","PeriodicalId":22152,"journal":{"name":"Surgery","volume":"179 ","pages":"Article 108942"},"PeriodicalIF":3.2000,"publicationDate":"2024-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Surgery","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S003960602400919X","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"SURGERY","Score":null,"Total":0}
引用次数: 0

Abstract

Background

Accurate case length estimation is a vital part of optimizing operating room use; however, significant inaccuracies exist with current solutions. The purpose of this study was to develop and validate an artificial intelligence system for improved surgical case length prediction by applying natural language processing and machine-learning methods.

Methods

All inpatient elective surgical cases longer than 30 minutes completed between 2017 and 2023 at a single, quaternary care hospital were considered. Data were split into training, test, and hold-out validation for model training and testing. Linear regression, CategoricalBoost, and feed-forward neural network each were trained and used embeddings created by bidirectional encoder representations from transformers or a bidirectional encoder representations from transformers model pretrained on clinical text. The average root mean squared error and mean absolute error were calculated for each model.

Results

A total of 125,493 cases were included. The highest performing model was the CategoricalBoost Regressor with bidirectional encoder representations from transformers model pretrained on clinical text embeddings (mean absolute error, 46.4 minutes), which was lower than the existing electronic health record estimates (120.0 minutes, P < 0.001). Accurate estimation of case length was defined as within ±20% of the actual case length with our model having 48% accuracy vs 17% accuracy for electronic health record–generated estimates.

Conclusion

An artificial intelligence model for surgical case length estimation outperforms existing institutional electronic health record predictions. On average, the estimate improved by 62% and approximately 2.8× the number of cases were correctly estimated. This study shows the successful development of machine learning models using advanced natural language processing techniques for improved surgical case length prediction.
用于手术病例长度预测的人工智能系统的开发和验证。
背景:准确的病例长度估计是优化手术室使用的重要组成部分;然而,目前的解决方案存在显著的不准确性。本研究的目的是开发和验证一个人工智能系统,通过应用自然语言处理和机器学习方法来改进手术病例长度预测。方法:选取2017年至2023年间在一家单一的第四护理医院完成的所有住院时间超过30分钟的选择性手术病例。数据被分成训练、测试和模型训练和测试的保留验证。线性回归、CategoricalBoost和前馈神经网络都经过了训练,并使用了由变压器的双向编码器表示或临床文本预训练的变压器模型的双向编码器表示创建的嵌入。计算每个模型的平均均方根误差和平均绝对误差。结果:共纳入125493例。表现最好的模型是具有双向编码器表示的CategoricalBoost回归模型,该模型来自临床文本嵌入预训练的变压器模型(平均绝对误差为46.4分钟),低于现有的电子健康记录估计(120.0分钟,P < 0.001)。病例长度的准确估计被定义为在实际病例长度的±20%以内,我们的模型准确率为48%,而电子健康记录生成估计的准确率为17%。结论:用于外科病例长度估计的人工智能模型优于现有的机构电子健康记录预测。平均而言,估计值提高了62%,正确估计的病例数约为2.8倍。这项研究显示了使用先进的自然语言处理技术的机器学习模型的成功开发,以改进手术病例长度预测。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Surgery
Surgery 医学-外科
CiteScore
5.40
自引率
5.30%
发文量
687
审稿时长
64 days
期刊介绍: For 66 years, Surgery has published practical, authoritative information about procedures, clinical advances, and major trends shaping general surgery. Each issue features original scientific contributions and clinical reports. Peer-reviewed articles cover topics in oncology, trauma, gastrointestinal, vascular, and transplantation surgery. The journal also publishes papers from the meetings of its sponsoring societies, the Society of University Surgeons, the Central Surgical Association, and the American Association of Endocrine Surgeons.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信