Effects of artificial intelligence assistance on endoscopist performance: Comparison of diagnostic performance in superficial esophageal squamous cell carcinoma detection using video-based models

IF 1.5 Q4 GASTROENTEROLOGY & HEPATOLOGY

DEN open Pub Date : 2025-05-02 DOI:10.1002/deo2.70083

Naoki Aoyama, Keiichiro Nakajo, Maasa Sasabe, Atsushi Inaba, Yuki Nakanishi, Hiroshi Seno, Tomonori Yano

{"title":"Effects of artificial intelligence assistance on endoscopist performance: Comparison of diagnostic performance in superficial esophageal squamous cell carcinoma detection using video-based models","authors":"Naoki Aoyama, Keiichiro Nakajo, Maasa Sasabe, Atsushi Inaba, Yuki Nakanishi, Hiroshi Seno, Tomonori Yano","doi":"10.1002/deo2.70083","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Objectives</h3>\n \n <p>Superficial esophageal squamous cell carcinoma (ESCC) detection is crucial. Although narrow-band imaging improves detection, its effectiveness is diminished by inexperienced endoscopists. The effects of artificial intelligence (AI) assistance on ESCC detection by endoscopists remain unclear. Therefore, this study aimed to develop and validate an AI model for ESCC detection using endoscopic video analysis and evaluate diagnostic improvements.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>Endoscopic videos with and without ESCC lesions were collected from May 2020 to January 2022. The AI model trained on annotated videos and 18 endoscopists (eight experts, 10 non-experts) evaluated their diagnostic performance. After 4 weeks, the endoscopists re-evaluated the test data with AI assistance. Sensitivity, specificity, and accuracy were compared between endoscopists with and without AI assistance.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>Training data comprised 280 cases (140 with and 140 without lesions), and test data, 115 cases (52 with and 63 without lesions). In the test data, the median lesion size was 14.5 mm (range: 1–100 mm), with pathological depths ranging from high-grade intraepithelial to submucosal neoplasia. The model's sensitivity, specificity, and accuracy were 76.0%, 79.4%, and 77.2%, respectively. With AI assistance, endoscopist sensitivity (57.4% vs. 66.5%) and accuracy (68.6% vs. 75.9%) improved significantly, while specificity increased slightly (87.0% vs. 91.6%). Experts demonstrated substantial improvements in sensitivity (59.1% vs. 70.0%) and accuracy (72.1% vs. 79.3%). Non-expert accuracy increased significantly (65.8% vs. 73.3%), with slight improvements in sensitivity (56.1% vs. 63.7%) and specificity (81.9% vs. 89.2%).</p>\n </section>\n \n <section>\n \n <h3> Conclusions</h3>\n \n <p>AI assistance enhances ESCC detection and improves endoscopists' diagnostic performance, regardless of experience.</p>\n </section>\n </div>","PeriodicalId":93973,"journal":{"name":"DEN open","volume":"6 1","pages":""},"PeriodicalIF":1.5000,"publicationDate":"2025-05-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/deo2.70083","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"DEN open","FirstCategoryId":"1085","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/deo2.70083","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"GASTROENTEROLOGY & HEPATOLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Objectives

Superficial esophageal squamous cell carcinoma (ESCC) detection is crucial. Although narrow-band imaging improves detection, its effectiveness is diminished by inexperienced endoscopists. The effects of artificial intelligence (AI) assistance on ESCC detection by endoscopists remain unclear. Therefore, this study aimed to develop and validate an AI model for ESCC detection using endoscopic video analysis and evaluate diagnostic improvements.

Methods

Endoscopic videos with and without ESCC lesions were collected from May 2020 to January 2022. The AI model trained on annotated videos and 18 endoscopists (eight experts, 10 non-experts) evaluated their diagnostic performance. After 4 weeks, the endoscopists re-evaluated the test data with AI assistance. Sensitivity, specificity, and accuracy were compared between endoscopists with and without AI assistance.

Results

Training data comprised 280 cases (140 with and 140 without lesions), and test data, 115 cases (52 with and 63 without lesions). In the test data, the median lesion size was 14.5 mm (range: 1–100 mm), with pathological depths ranging from high-grade intraepithelial to submucosal neoplasia. The model's sensitivity, specificity, and accuracy were 76.0%, 79.4%, and 77.2%, respectively. With AI assistance, endoscopist sensitivity (57.4% vs. 66.5%) and accuracy (68.6% vs. 75.9%) improved significantly, while specificity increased slightly (87.0% vs. 91.6%). Experts demonstrated substantial improvements in sensitivity (59.1% vs. 70.0%) and accuracy (72.1% vs. 79.3%). Non-expert accuracy increased significantly (65.8% vs. 73.3%), with slight improvements in sensitivity (56.1% vs. 63.7%) and specificity (81.9% vs. 89.2%).

Conclusions

AI assistance enhances ESCC detection and improves endoscopists' diagnostic performance, regardless of experience.

Abstract Image

查看原文本刊更多论文

人工智能辅助对内镜医师表现的影响：基于视频模型的浅表食管鳞状细胞癌诊断表现的比较

目的浅表食管鳞状细胞癌（ESCC）的检测至关重要。虽然窄带成像改善了检测，但由于内窥镜医师缺乏经验，其有效性会降低。人工智能（AI）辅助内镜医师检测ESCC的影响尚不清楚。因此，本研究旨在通过内镜视频分析开发并验证用于ESCC检测的AI模型，并评估诊断改进。方法收集2020年5月至2022年1月有和无ESCC病变的内镜视频。人工智能模型接受了注释视频的训练，18名内窥镜医生（8名专家，10名非专家）评估了他们的诊断表现。4周后，内窥镜医师在人工智能辅助下重新评估测试数据。在有和没有人工智能辅助的情况下比较内窥镜医师的敏感性、特异性和准确性。结果训练数据为280例（有病变140例，无病变140例），测试数据为115例（有病变52例，无病变63例）。在试验数据中，中位病变大小为14.5 mm（范围：1-100 mm），病理深度从高级别上皮内瘤变到粘膜下瘤变不等。该模型的敏感性、特异性和准确性分别为76.0%、79.4%和77.2%。在人工智能辅助下，内镜医师的敏感性（57.4% vs. 66.5%）和准确性（68.6% vs. 75.9%）显著提高，特异性略有提高（87.0% vs. 91.6%）。专家证明了灵敏度（59.1% vs. 70.0%）和准确性（72.1% vs. 79.3%）的显著提高。非专家准确度显著提高（65.8% vs. 73.3%），敏感性（56.1% vs. 63.7%）和特异性（81.9% vs. 89.2%）略有改善。结论人工智能辅助增强了ESCC的检测，提高了内镜医师的诊断能力，无论经验如何。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

DEN open

CiteScore

1.30

自引率

0.00%

发文量