Thorax computed tomography (CTX) guided ground truth annotation of CHEST radiographs (CXR) for improved classification and detection of COVID-19

IF 2.2 4区 医学 Q3 ENGINEERING, BIOMEDICAL
Şükrü Mehmet Ertürk, Tuğçe Toprak, Rana Günöz Cömert, Cemre Candemir, Eda Cingöz, Zeynep Nur Akyol Sari, Celal Caner Ercan, Esin Düvek, Berke Ersoy, Edanur Karapinar, Atadan Tunaci, M. Alper Selver
{"title":"Thorax computed tomography (CTX) guided ground truth annotation of CHEST radiographs (CXR) for improved classification and detection of COVID-19","authors":"Şükrü Mehmet Ertürk,&nbsp;Tuğçe Toprak,&nbsp;Rana Günöz Cömert,&nbsp;Cemre Candemir,&nbsp;Eda Cingöz,&nbsp;Zeynep Nur Akyol Sari,&nbsp;Celal Caner Ercan,&nbsp;Esin Düvek,&nbsp;Berke Ersoy,&nbsp;Edanur Karapinar,&nbsp;Atadan Tunaci,&nbsp;M. Alper Selver","doi":"10.1002/cnm.3823","DOIUrl":null,"url":null,"abstract":"<p>Several data sets have been collected and various artificial intelligence models have been developed for COVID-19 classification and detection from both chest radiography (CXR) and thorax computed tomography (CTX) images. However, the pitfalls and shortcomings of these systems significantly limit their clinical use. In this respect, improving the weaknesses of advanced models can be very effective besides developing new ones. The inability to diagnose ground-glass opacities by conventional CXR has limited the use of this modality in the diagnostic work-up of COVID-19. In our study, we investigated whether we could increase the diagnostic efficiency by collecting a novel CXR data set, which contains pneumonic regions that are not visible to the experts and can only be annotated under CTX guidance. We develop an ensemble methodology of well-established deep CXR models for this new data set and develop a machine learning-based non-maximum suppression strategy to boost the performance for challenging CXR images. CTX and CXR images of 379 patients who applied to our hospital with suspected COVID-19 were evaluated with consensus by seven radiologists. Among these, CXR images of 161 patients who also have had a CTX examination on the same day or until the day before or after and whose CTX findings are compatible with COVID-19 pneumonia, are selected for annotating. CTX images are arranged in the main section passing through the anterior, middle, and posterior according to the sagittal plane with the reformed maximum intensity projection (MIP) method in the coronal plane. Based on the analysis of coronal MIP reconstructed CTX images, the regions corresponding to the pneumonia foci are annotated manually in CXR images. Radiologically classified posterior to anterior (PA) CXR of 218 patients with negative thorax CTX imaging were classified as COVID-19 pneumonia negative group. Accordingly, we have collected a new data set using anonymized CXR (JPEG) and CT (DICOM) images, where the PA CXRs contain pneumonic regions that are hidden or not easily recognized and annotated under CTX guidance. The reference finding was the presence of pneumonic infiltration consistent with COVID-19 on chest CTX examination. COVID-Net, a specially designed convolutional neural network, was used to detect cases of COVID-19 among CXRs. Diagnostic performances were evaluated by ROC analysis by applying six COVID-Net variants (COVIDNet-CXR3-A, -B, -C/COVIDNet-CXR4-A, -B, -C) to the defined data set and combining these models in various ways via ensemble strategies. Finally, a convex optimization strategy is carried out to find the outperforming weighted ensemble of individual models. The mean age of 161 patients with pneumonia was 49.31 ± 15.12, and the median age was 48 years. The mean age of 218 patients without signs of pneumonia in thorax CTX examination was 40.04 ± 14.46, and the median was 38. When working with different combinations of COVID-Net's six variants, the area under the curve (AUC) using the ensemble COVID-Net CXR 4A-4B-3C was .78, sensitivity 67%, specificity 95%; COVID-Net CXR 4a-3b-3c was .79, sensitivity 69% and specificity 94%. When diverse and complementary COVID-Net models are used together through an ensemble, it has been determined that the AUC values are close to other studies, and the specificity is significantly higher than other studies in the literature.</p>","PeriodicalId":50349,"journal":{"name":"International Journal for Numerical Methods in Biomedical Engineering","volume":"40 6","pages":""},"PeriodicalIF":2.2000,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cnm.3823","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal for Numerical Methods in Biomedical Engineering","FirstCategoryId":"5","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/cnm.3823","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"ENGINEERING, BIOMEDICAL","Score":null,"Total":0}
引用次数: 0

Abstract

Several data sets have been collected and various artificial intelligence models have been developed for COVID-19 classification and detection from both chest radiography (CXR) and thorax computed tomography (CTX) images. However, the pitfalls and shortcomings of these systems significantly limit their clinical use. In this respect, improving the weaknesses of advanced models can be very effective besides developing new ones. The inability to diagnose ground-glass opacities by conventional CXR has limited the use of this modality in the diagnostic work-up of COVID-19. In our study, we investigated whether we could increase the diagnostic efficiency by collecting a novel CXR data set, which contains pneumonic regions that are not visible to the experts and can only be annotated under CTX guidance. We develop an ensemble methodology of well-established deep CXR models for this new data set and develop a machine learning-based non-maximum suppression strategy to boost the performance for challenging CXR images. CTX and CXR images of 379 patients who applied to our hospital with suspected COVID-19 were evaluated with consensus by seven radiologists. Among these, CXR images of 161 patients who also have had a CTX examination on the same day or until the day before or after and whose CTX findings are compatible with COVID-19 pneumonia, are selected for annotating. CTX images are arranged in the main section passing through the anterior, middle, and posterior according to the sagittal plane with the reformed maximum intensity projection (MIP) method in the coronal plane. Based on the analysis of coronal MIP reconstructed CTX images, the regions corresponding to the pneumonia foci are annotated manually in CXR images. Radiologically classified posterior to anterior (PA) CXR of 218 patients with negative thorax CTX imaging were classified as COVID-19 pneumonia negative group. Accordingly, we have collected a new data set using anonymized CXR (JPEG) and CT (DICOM) images, where the PA CXRs contain pneumonic regions that are hidden or not easily recognized and annotated under CTX guidance. The reference finding was the presence of pneumonic infiltration consistent with COVID-19 on chest CTX examination. COVID-Net, a specially designed convolutional neural network, was used to detect cases of COVID-19 among CXRs. Diagnostic performances were evaluated by ROC analysis by applying six COVID-Net variants (COVIDNet-CXR3-A, -B, -C/COVIDNet-CXR4-A, -B, -C) to the defined data set and combining these models in various ways via ensemble strategies. Finally, a convex optimization strategy is carried out to find the outperforming weighted ensemble of individual models. The mean age of 161 patients with pneumonia was 49.31 ± 15.12, and the median age was 48 years. The mean age of 218 patients without signs of pneumonia in thorax CTX examination was 40.04 ± 14.46, and the median was 38. When working with different combinations of COVID-Net's six variants, the area under the curve (AUC) using the ensemble COVID-Net CXR 4A-4B-3C was .78, sensitivity 67%, specificity 95%; COVID-Net CXR 4a-3b-3c was .79, sensitivity 69% and specificity 94%. When diverse and complementary COVID-Net models are used together through an ensemble, it has been determined that the AUC values are close to other studies, and the specificity is significantly higher than other studies in the literature.

Abstract Image

Abstract Image

胸部计算机断层扫描(CTX)引导下的CHEST射线照片(CXR)地面实况标注,用于改进COVID-19的分类和检测
目前已收集了多个数据集,并开发了各种人工智能模型,用于从胸部放射摄影(CXR)和胸部计算机断层扫描(CTX)图像中进行 COVID-19 分类和检测。然而,这些系统的缺陷和不足极大地限制了它们在临床上的应用。在这方面,除了开发新的模型外,改进先进模型的弱点也非常有效。传统的 CXR 无法诊断磨玻璃不透明,这限制了这种模式在 COVID-19 诊断工作中的应用。在我们的研究中,我们研究了是否可以通过收集新的 CXR 数据集来提高诊断效率,该数据集包含了专家无法看到的、只能在 CTX 引导下标注的肺部区域。我们针对这一新数据集开发了一种成熟的深度 CXR 模型集合方法,并开发了一种基于机器学习的非最大抑制策略,以提高具有挑战性的 CXR 图像的性能。七位放射科专家对 379 名疑似 COVID-19 患者的 CTX 和 CXR 图像进行了一致评估。其中,161 名患者在同一天或前后一天也接受了 CTX 检查,且 CTX 结果与 COVID-19 肺炎相符,这些患者的 CXR 图像被选中进行注释。根据矢状面将 CTX 图像排列在经过前、中、后的主切面上,并在冠状面上采用改良最大强度投影(MIP)方法。根据对冠状面 MIP 重建 CTX 图像的分析,在 CXR 图像中手动标注与肺炎病灶相对应的区域。218 名胸部 CTX 成像为阴性的患者经放射学分类的后向前(PA)CXR 被归入 COVID-19 肺炎阴性组。因此,我们利用匿名 CXR(JPEG)和 CT(DICOM)图像收集了一组新数据,其中 PA CXR 包含隐藏的或在 CTX 引导下不易识别和标注的肺炎区域。参考发现是胸部 CTX 检查中出现与 COVID-19 一致的肺炎浸润。COVID-Net 是一种专门设计的卷积神经网络,用于检测 CXR 中的 COVID-19 病例。通过对定义的数据集应用六种 COVID-Net 变体(COVIDNet-CXR3-A、-B、-C/COVIDNet-CXR4-A、-B、-C),并通过集合策略以不同方式组合这些模型,以 ROC 分析评估诊断性能。最后,通过凸优化策略,找到性能更优的加权集合单个模型。161 名肺炎患者的平均年龄为 49.31±15.12 岁,中位年龄为 48 岁。218 名胸部 CTX 检查无肺炎症状的患者的平均年龄为(40.04 ± 14.46)岁,中位数为 38 岁。在使用 COVID-Net 六种变体的不同组合时,COVID-Net CXR 4A-4B-3C 组合的曲线下面积(AUC)为 0.78,灵敏度为 67%,特异性为 95%;COVID-Net CXR 4a-3b-3c 的曲线下面积(AUC)为 0.79,灵敏度为 69%,特异性为 94%。通过组合使用不同的互补 COVID-Net 模型,可以确定 AUC 值与其他研究接近,特异性明显高于文献中的其他研究。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
International Journal for Numerical Methods in Biomedical Engineering
International Journal for Numerical Methods in Biomedical Engineering ENGINEERING, BIOMEDICAL-MATHEMATICAL & COMPUTATIONAL BIOLOGY
CiteScore
4.50
自引率
9.50%
发文量
103
审稿时长
3 months
期刊介绍: All differential equation based models for biomedical applications and their novel solutions (using either established numerical methods such as finite difference, finite element and finite volume methods or new numerical methods) are within the scope of this journal. Manuscripts with experimental and analytical themes are also welcome if a component of the paper deals with numerical methods. Special cases that may not involve differential equations such as image processing, meshing and artificial intelligence are within the scope. Any research that is broadly linked to the wellbeing of the human body, either directly or indirectly, is also within the scope of this journal.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信