David Roser, Michael Meinikheim, Anna Muzalyova, Robert Mendel, Christoph Palm, Andreas Probst, Sandra Nagl, Markus W. Scheppach, Christoph Römmele, Elisabeth Schnoy, Nasim Parsa, Michael F. Byrne, Helmut Messmann, Alanna Ebigbo
{"title":"人工智能辅助内窥镜检查与检查员信心：巴雷特食管人-人工智能互动研究（附视频）","authors":"David Roser, Michael Meinikheim, Anna Muzalyova, Robert Mendel, Christoph Palm, Andreas Probst, Sandra Nagl, Markus W. Scheppach, Christoph Römmele, Elisabeth Schnoy, Nasim Parsa, Michael F. Byrne, Helmut Messmann, Alanna Ebigbo","doi":"10.1002/deo2.70150","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Objective</h3>\n \n <p>Despite high stand-alone performance, studies demonstrate that artificial intelligence (AI)-supported endoscopic diagnostics often fall short in clinical applications due to human-AI interaction factors. This video-based trial on Barrett's esophagus aimed to investigate how examiner behavior, their levels of confidence, and system usability influence the diagnostic outcomes of AI-assisted endoscopy.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>The present analysis employed data from a multicenter randomized controlled tandem video trial involving 22 endoscopists with varying degrees of expertise. Participants were tasked with evaluating a set of 96 endoscopic videos of Barrett's esophagus in two distinct rounds, with and without AI assistance. Diagnostic confidence levels were recorded, and decision changes were categorized according to the AI prediction. Additional surveys assessed user experience and system usability ratings.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>AI assistance significantly increased examiner confidence levels (<i>p</i> < 0.001) and accuracy. Withdrawing AI assistance decreased confidence (<i>p</i> < 0.001), but not accuracy. Experts consistently reported higher confidence than non-experts (<i>p</i> < 0.001), regardless of performance. Despite improved confidence, correct AI guidance was disregarded in 16% of all cases, and 9% of initially correct diagnoses were changed to incorrect ones. Overreliance on AI, algorithm aversion, and uncertainty in AI predictions were identified as key factors influencing outcomes. The System Usability Scale questionnaire scores indicated good to excellent usability, with non-experts scoring 73.5 and experts 85.6.</p>\n </section>\n \n <section>\n \n <h3> Conclusions</h3>\n \n <p>Our findings highlight the pivotal function of examiner behavior in AI-assisted endoscopy. To fully realize the benefits of AI, implementing explainable AI, improving user interfaces, and providing targeted training are essential. Addressing these factors could enhance diagnostic accuracy and confidence in clinical practice.</p>\n </section>\n </div>","PeriodicalId":93973,"journal":{"name":"DEN open","volume":"6 1","pages":""},"PeriodicalIF":1.5000,"publicationDate":"2025-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/deo2.70150","citationCount":"0","resultStr":"{\"title\":\"Artificial Intelligence-assisted Endoscopy and Examiner Confidence: A Study on Human–Artificial Intelligence Interaction in Barrett's Esophagus (With Video)\",\"authors\":\"David Roser, Michael Meinikheim, Anna Muzalyova, Robert Mendel, Christoph Palm, Andreas Probst, Sandra Nagl, Markus W. Scheppach, Christoph Römmele, Elisabeth Schnoy, Nasim Parsa, Michael F. Byrne, Helmut Messmann, Alanna Ebigbo\",\"doi\":\"10.1002/deo2.70150\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div>\\n \\n \\n <section>\\n \\n <h3> Objective</h3>\\n \\n <p>Despite high stand-alone performance, studies demonstrate that artificial intelligence (AI)-supported endoscopic diagnostics often fall short in clinical applications due to human-AI interaction factors. This video-based trial on Barrett's esophagus aimed to investigate how examiner behavior, their levels of confidence, and system usability influence the diagnostic outcomes of AI-assisted endoscopy.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Methods</h3>\\n \\n <p>The present analysis employed data from a multicenter randomized controlled tandem video trial involving 22 endoscopists with varying degrees of expertise. Participants were tasked with evaluating a set of 96 endoscopic videos of Barrett's esophagus in two distinct rounds, with and without AI assistance. Diagnostic confidence levels were recorded, and decision changes were categorized according to the AI prediction. Additional surveys assessed user experience and system usability ratings.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Results</h3>\\n \\n <p>AI assistance significantly increased examiner confidence levels (<i>p</i> < 0.001) and accuracy. Withdrawing AI assistance decreased confidence (<i>p</i> < 0.001), but not accuracy. Experts consistently reported higher confidence than non-experts (<i>p</i> < 0.001), regardless of performance. Despite improved confidence, correct AI guidance was disregarded in 16% of all cases, and 9% of initially correct diagnoses were changed to incorrect ones. Overreliance on AI, algorithm aversion, and uncertainty in AI predictions were identified as key factors influencing outcomes. The System Usability Scale questionnaire scores indicated good to excellent usability, with non-experts scoring 73.5 and experts 85.6.</p>\\n </section>\\n \\n <section>\\n \\n <h3> Conclusions</h3>\\n \\n <p>Our findings highlight the pivotal function of examiner behavior in AI-assisted endoscopy. To fully realize the benefits of AI, implementing explainable AI, improving user interfaces, and providing targeted training are essential. Addressing these factors could enhance diagnostic accuracy and confidence in clinical practice.</p>\\n </section>\\n </div>\",\"PeriodicalId\":93973,\"journal\":{\"name\":\"DEN open\",\"volume\":\"6 1\",\"pages\":\"\"},\"PeriodicalIF\":1.5000,\"publicationDate\":\"2025-06-19\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://onlinelibrary.wiley.com/doi/epdf/10.1002/deo2.70150\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"DEN open\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://onlinelibrary.wiley.com/doi/10.1002/deo2.70150\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q4\",\"JCRName\":\"GASTROENTEROLOGY & HEPATOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"DEN open","FirstCategoryId":"1085","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/deo2.70150","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"GASTROENTEROLOGY & HEPATOLOGY","Score":null,"Total":0}
Artificial Intelligence-assisted Endoscopy and Examiner Confidence: A Study on Human–Artificial Intelligence Interaction in Barrett's Esophagus (With Video)
Objective
Despite high stand-alone performance, studies demonstrate that artificial intelligence (AI)-supported endoscopic diagnostics often fall short in clinical applications due to human-AI interaction factors. This video-based trial on Barrett's esophagus aimed to investigate how examiner behavior, their levels of confidence, and system usability influence the diagnostic outcomes of AI-assisted endoscopy.
Methods
The present analysis employed data from a multicenter randomized controlled tandem video trial involving 22 endoscopists with varying degrees of expertise. Participants were tasked with evaluating a set of 96 endoscopic videos of Barrett's esophagus in two distinct rounds, with and without AI assistance. Diagnostic confidence levels were recorded, and decision changes were categorized according to the AI prediction. Additional surveys assessed user experience and system usability ratings.
Results
AI assistance significantly increased examiner confidence levels (p < 0.001) and accuracy. Withdrawing AI assistance decreased confidence (p < 0.001), but not accuracy. Experts consistently reported higher confidence than non-experts (p < 0.001), regardless of performance. Despite improved confidence, correct AI guidance was disregarded in 16% of all cases, and 9% of initially correct diagnoses were changed to incorrect ones. Overreliance on AI, algorithm aversion, and uncertainty in AI predictions were identified as key factors influencing outcomes. The System Usability Scale questionnaire scores indicated good to excellent usability, with non-experts scoring 73.5 and experts 85.6.
Conclusions
Our findings highlight the pivotal function of examiner behavior in AI-assisted endoscopy. To fully realize the benefits of AI, implementing explainable AI, improving user interfaces, and providing targeted training are essential. Addressing these factors could enhance diagnostic accuracy and confidence in clinical practice.
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
