用于腹腔镜手术认知工作量分类的新型多模态传感和机器学习策略。

IF 3.5 2区 医学 Q2 ONCOLOGY
Ejso Pub Date : 2024-10-15 DOI:10.1016/j.ejso.2024.108735
Ravi Naik, Adrian Rubio-Solis, Kaizhe Jin, George Mylonas
{"title":"用于腹腔镜手术认知工作量分类的新型多模态传感和机器学习策略。","authors":"Ravi Naik, Adrian Rubio-Solis, Kaizhe Jin, George Mylonas","doi":"10.1016/j.ejso.2024.108735","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Surgeons can experience elevated cognitive workload (CWL) during surgery due to various factors including operative technicalities and the environmental demands of the operating theatre. This can result in poorer outcomes and have a detrimental effect on surgeon well-being. The objective measurement of CWL provides a potential solution to facilitate classification of workload levels, however results are variable when physiological measures are used in isolation. The aim of this study is to develop and propose a multimodal machine learning (ML) approach to classify CWL levels using a bespoke sensor platform and to develop a ML approach to impute missing pupil diameter measures due to the effect of blinking or noise.</p><p><strong>Materials and methods: </strong>Ten surgical trainees performed a simulated laparoscopic cholecystectomy under cognitive conditions of increasing difficulty, namely a modified auditory N-back task with increasing difficulty and a verbal clinical scenario. Physiological measures were recorded using a novel platform (MAESTRO). Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) were used as direct measures of CWL. Indirect measures included electromyography (EMG), electrocardiography (ECG) and pupil diameter (PD). A reference point for validation was provided by subjective assessment of perceived CWL using the SURG-TLX. A multimodal machine learning approach that systematically implements a CNN-BiLSTM, a binary version of the metaheuristic Manta Ray Foraging Optimisation (BMRFO) and a version of Fuzzy C-Means (FCM) called Optimal Completion Strategy (OCS) was used to classify the associated perceived CWL state.</p><p><strong>Results: </strong>Compared to other state of the art classification techniques, cross-validation results for the classification of CWL levels suggest that the CNN-BLSTM and BMRFO approach provides an average accuracy of 97 % based on the confusion matrix. Additionally, OCS demonstrated a superior average performance of 9.15 % in terms of Root-Mean-Square-Error (RMSE) when compared to other PD imputation methods.</p><p><strong>Conclusion: </strong>Perceived CWL levels were correctly classified using a multimodal ML approach. This approach provides a potential route to accurately classify CWL levels, which may have application in future surgical training and assessment programs as well as the development of cognitive support systems in the operating room.</p>","PeriodicalId":11522,"journal":{"name":"Ejso","volume":" ","pages":"108735"},"PeriodicalIF":3.5000,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Novel multimodal sensing and machine learning strategies to classify cognitive workload in laparoscopic surgery.\",\"authors\":\"Ravi Naik, Adrian Rubio-Solis, Kaizhe Jin, George Mylonas\",\"doi\":\"10.1016/j.ejso.2024.108735\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Background: </strong>Surgeons can experience elevated cognitive workload (CWL) during surgery due to various factors including operative technicalities and the environmental demands of the operating theatre. This can result in poorer outcomes and have a detrimental effect on surgeon well-being. The objective measurement of CWL provides a potential solution to facilitate classification of workload levels, however results are variable when physiological measures are used in isolation. The aim of this study is to develop and propose a multimodal machine learning (ML) approach to classify CWL levels using a bespoke sensor platform and to develop a ML approach to impute missing pupil diameter measures due to the effect of blinking or noise.</p><p><strong>Materials and methods: </strong>Ten surgical trainees performed a simulated laparoscopic cholecystectomy under cognitive conditions of increasing difficulty, namely a modified auditory N-back task with increasing difficulty and a verbal clinical scenario. Physiological measures were recorded using a novel platform (MAESTRO). Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) were used as direct measures of CWL. Indirect measures included electromyography (EMG), electrocardiography (ECG) and pupil diameter (PD). A reference point for validation was provided by subjective assessment of perceived CWL using the SURG-TLX. A multimodal machine learning approach that systematically implements a CNN-BiLSTM, a binary version of the metaheuristic Manta Ray Foraging Optimisation (BMRFO) and a version of Fuzzy C-Means (FCM) called Optimal Completion Strategy (OCS) was used to classify the associated perceived CWL state.</p><p><strong>Results: </strong>Compared to other state of the art classification techniques, cross-validation results for the classification of CWL levels suggest that the CNN-BLSTM and BMRFO approach provides an average accuracy of 97 % based on the confusion matrix. Additionally, OCS demonstrated a superior average performance of 9.15 % in terms of Root-Mean-Square-Error (RMSE) when compared to other PD imputation methods.</p><p><strong>Conclusion: </strong>Perceived CWL levels were correctly classified using a multimodal ML approach. This approach provides a potential route to accurately classify CWL levels, which may have application in future surgical training and assessment programs as well as the development of cognitive support systems in the operating room.</p>\",\"PeriodicalId\":11522,\"journal\":{\"name\":\"Ejso\",\"volume\":\" \",\"pages\":\"108735\"},\"PeriodicalIF\":3.5000,\"publicationDate\":\"2024-10-15\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Ejso\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1016/j.ejso.2024.108735\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ONCOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Ejso","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1016/j.ejso.2024.108735","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ONCOLOGY","Score":null,"Total":0}
引用次数: 0

摘要

背景:由于手术技术和手术室环境要求等各种因素,外科医生在手术过程中会出现认知工作量(CWL)增加的情况。这可能会导致较差的结果,并对外科医生的健康产生不利影响。对 CWL 的客观测量提供了一个潜在的解决方案,便于对工作量水平进行分类,但如果单独使用生理测量方法,结果会有差异。本研究的目的是开发并提出一种多模态机器学习(ML)方法,利用定制的传感器平台对 CWL 水平进行分类,并开发一种 ML 方法来弥补因眨眼或噪音影响而缺失的瞳孔直径测量值:十名外科学员在难度递增的认知条件下进行了模拟腹腔镜胆囊切除术,即难度递增的改良听觉N-back任务和口头临床情景。使用新型平台(MAESTRO)记录生理指标。脑电图(EEG)和功能性近红外光谱(fNIRS)被用作 CWL 的直接测量指标。间接测量包括肌电图(EMG)、心电图(ECG)和瞳孔直径(PD)。使用 SURG-TLX 对感知的 CWL 进行主观评估,为验证提供了参考点。系统实施了 CNN-BiLSTM、二进制版本的元启发式 Manta Ray Foraging Optimisation (BMRFO) 和称为 Optimal Completion Strategy (OCS) 的模糊 C-Means (FCM) 版本的多模态机器学习方法,用于对相关的感知 CWL 状态进行分类:与其他最先进的分类技术相比,CWL 等级分类的交叉验证结果表明,根据混淆矩阵,CNN-BLSTM 和 BMRFO 方法的平均准确率为 97%。此外,与其他 PD 估算方法相比,OCS 的均方根误差(RMSE)为 9.15%,表现更优:结论:使用多模态 ML 方法对感知的 CWL 水平进行了正确分类。这种方法提供了准确分类 CWL 水平的潜在途径,可应用于未来的外科培训和评估计划,以及手术室认知支持系统的开发。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Novel multimodal sensing and machine learning strategies to classify cognitive workload in laparoscopic surgery.

Background: Surgeons can experience elevated cognitive workload (CWL) during surgery due to various factors including operative technicalities and the environmental demands of the operating theatre. This can result in poorer outcomes and have a detrimental effect on surgeon well-being. The objective measurement of CWL provides a potential solution to facilitate classification of workload levels, however results are variable when physiological measures are used in isolation. The aim of this study is to develop and propose a multimodal machine learning (ML) approach to classify CWL levels using a bespoke sensor platform and to develop a ML approach to impute missing pupil diameter measures due to the effect of blinking or noise.

Materials and methods: Ten surgical trainees performed a simulated laparoscopic cholecystectomy under cognitive conditions of increasing difficulty, namely a modified auditory N-back task with increasing difficulty and a verbal clinical scenario. Physiological measures were recorded using a novel platform (MAESTRO). Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) were used as direct measures of CWL. Indirect measures included electromyography (EMG), electrocardiography (ECG) and pupil diameter (PD). A reference point for validation was provided by subjective assessment of perceived CWL using the SURG-TLX. A multimodal machine learning approach that systematically implements a CNN-BiLSTM, a binary version of the metaheuristic Manta Ray Foraging Optimisation (BMRFO) and a version of Fuzzy C-Means (FCM) called Optimal Completion Strategy (OCS) was used to classify the associated perceived CWL state.

Results: Compared to other state of the art classification techniques, cross-validation results for the classification of CWL levels suggest that the CNN-BLSTM and BMRFO approach provides an average accuracy of 97 % based on the confusion matrix. Additionally, OCS demonstrated a superior average performance of 9.15 % in terms of Root-Mean-Square-Error (RMSE) when compared to other PD imputation methods.

Conclusion: Perceived CWL levels were correctly classified using a multimodal ML approach. This approach provides a potential route to accurately classify CWL levels, which may have application in future surgical training and assessment programs as well as the development of cognitive support systems in the operating room.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Ejso
Ejso 医学-外科
CiteScore
6.40
自引率
2.60%
发文量
1148
审稿时长
41 days
期刊介绍: JSO - European Journal of Surgical Oncology ("the Journal of Cancer Surgery") is the Official Journal of the European Society of Surgical Oncology and BASO ~ the Association for Cancer Surgery. The EJSO aims to advance surgical oncology research and practice through the publication of original research articles, review articles, editorials, debates and correspondence.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信