Ioannis Kafetzis, Philipp Sodmann, Bianca-Elena Herghelegiu, Markus Brand, Wolfram G Zoller, Florian Seyfried, Karl-Hermann Fuchs, Alexander Meining, Alexander Hann
{"title":"Prospective Evaluation of Real-Time Artificial Intelligence for the Hill Classification of the Gastroesophageal Junction.","authors":"Ioannis Kafetzis, Philipp Sodmann, Bianca-Elena Herghelegiu, Markus Brand, Wolfram G Zoller, Florian Seyfried, Karl-Hermann Fuchs, Alexander Meining, Alexander Hann","doi":"10.1002/ueg2.12721","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Assessment of the gastroesophageal junction (GEJ) is an integral part of gastroscopy; however, the absence of standardized reporting hinders consistency of examination documentation. The Hill classification offers a standardized approach for evaluating the GEJ. This study aims to compare the accuracy of an artificial intelligence (AI) system with that of physicians in classifying the GEJ according to Hill in a prospective, blinded, superiority trial.</p><p><strong>Methods: </strong>Consecutive patients scheduled for gastroscopy with an intact GEJ were recruited during clinical routine from October 2023 to December 2023. Nine physicians (six experienced, three inexperienced) assessed the Hill grade, and the AI system operated in the background in real-time. The gold standard was determined by a majority vote of independent assessments by three expert endoscopists who did not participate in the study. The primary outcome was accuracy. Secondary outcomes were per-Hill grade analysis and result comparison for experienced and inexperienced endoscopists separately.</p><p><strong>Results: </strong>In 131 analysed examinations the AI's accuracy of 84.7% (95% CI: 78.6-90.8) was significantly higher than 62.5% (95% CI: 54.2-71) of physicians (p < 0.01). The AI outperformed physicians in all but one cases in the per-Hill-class analysis. AI was significantly more accurate than inexperienced physicians (85% vs. 56%, p < 0.01) and in trend better than experienced physicians (84% vs. 69.6%, p = 0.07).</p><p><strong>Conclusions: </strong>AI was significantly more accurate than examiners in assessing the Hill classification. This superior model performance can prove beneficial for endoscopists, especially those with limited experience.</p><p><strong>Trial registration: </strong>ClinicalTrials.gov identifier: NCT06040723.</p>","PeriodicalId":23444,"journal":{"name":"United European Gastroenterology Journal","volume":" ","pages":""},"PeriodicalIF":5.8000,"publicationDate":"2024-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"United European Gastroenterology Journal","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1002/ueg2.12721","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"GASTROENTEROLOGY & HEPATOLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
Background: Assessment of the gastroesophageal junction (GEJ) is an integral part of gastroscopy; however, the absence of standardized reporting hinders consistency of examination documentation. The Hill classification offers a standardized approach for evaluating the GEJ. This study aims to compare the accuracy of an artificial intelligence (AI) system with that of physicians in classifying the GEJ according to Hill in a prospective, blinded, superiority trial.
Methods: Consecutive patients scheduled for gastroscopy with an intact GEJ were recruited during clinical routine from October 2023 to December 2023. Nine physicians (six experienced, three inexperienced) assessed the Hill grade, and the AI system operated in the background in real-time. The gold standard was determined by a majority vote of independent assessments by three expert endoscopists who did not participate in the study. The primary outcome was accuracy. Secondary outcomes were per-Hill grade analysis and result comparison for experienced and inexperienced endoscopists separately.
Results: In 131 analysed examinations the AI's accuracy of 84.7% (95% CI: 78.6-90.8) was significantly higher than 62.5% (95% CI: 54.2-71) of physicians (p < 0.01). The AI outperformed physicians in all but one cases in the per-Hill-class analysis. AI was significantly more accurate than inexperienced physicians (85% vs. 56%, p < 0.01) and in trend better than experienced physicians (84% vs. 69.6%, p = 0.07).
Conclusions: AI was significantly more accurate than examiners in assessing the Hill classification. This superior model performance can prove beneficial for endoscopists, especially those with limited experience.
期刊介绍:
United European Gastroenterology Journal (UEG Journal) is the official Journal of the United European Gastroenterology (UEG), a professional non-profit organisation combining all the leading European societies concerned with digestive disease. UEG’s member societies represent over 22,000 specialists working across medicine, surgery, paediatrics, GI oncology and endoscopy, which makes UEG a unique platform for collaboration and the exchange of knowledge.