Inter-rater concordance in the classification of COVID-19 in chest X-ray images using the RANZCR template for COVID-19 infection

IF 1.3 Q3 RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING

Journal of Medical Imaging and Radiation Sciences Pub Date : 2025-05-01 DOI:10.1016/j.jmir.2025.101911

Peter O’Reilly , Dania Abu Awwad , Sarah Lewis , Warren Reed , Ernest Ekpo

{"title":"Inter-rater concordance in the classification of COVID-19 in chest X-ray images using the RANZCR template for COVID-19 infection","authors":"Peter O’Reilly , Dania Abu Awwad , Sarah Lewis , Warren Reed , Ernest Ekpo","doi":"10.1016/j.jmir.2025.101911","DOIUrl":null,"url":null,"abstract":"<div><h3>Introduction</h3><div>The Royal Australian and New Zealand College of Radiologists (RANZCR) has developed a reporting template to assist in the categorization of COVID-19 in chest X-ray (CXR) images and the levels of COVID-19 infection. Whilst CXRs are reported by radiologists, radiographers are often the first to assess the CXRs, and have the potential to support immediate triaging of patients with COVID-19. However, inter-reader concordance in the use of this reporting template remains underexplored.</div></div><div><h3>Methods</h3><div>70 CXR examinations comprising of the four categories in the RANZCR chest X-ray (CXR) COVID-19 reporting template were used for the study. These included: ‘typical’ (for COVID-19) (<em>n</em> = 30); ‘indeterminate’ (for COVID-19) (<em>n</em> = 20); ‘other diagnoses favoured’ (<em>n</em> = 10) and ‘normal’ (<em>n</em> = 10). These images were independently categorised using the RANZCR reporting template by three cohorts of readers: 12 radiologists, 13 registered radiographers, and 12 final-year radiography students. A Weighted Kappa (κ<sub>w</sub>) was used to evaluate inter-reader agreement within and between the cohort of readers.</div></div><div><h3>Results</h3><div>Radiologists demonstrated fair (κ<em><sub>w</sub></em> = 0.32) to substantial (κ<em><sub>w</sub></em> = 0.77) inter-reader agreement, and their overall inter-reader was moderate (κ<em><sub>w</sub></em> = 0.56). Registered radiographers demonstrated no (κ<em><sub>w</sub></em> = -0.01) to moderate agreement (κ<em><sub>w</sub></em> = 0.59), and their overall agreement was fair (κ<em><sub>w</sub></em> = 0.31). Fourth year student radiographers demonstrated slight (κ<em><sub>w</sub></em> = 0.004) to substantial (κ<sub>w</sub> <sub>=</sub> 0.8) agreement, with a moderate (κ<em><sub>w</sub></em> = 0.47) overall agreement among final year student radiographers.</div></div><div><h3>Conclusion</h3><div>There are wide variations in the classification of the CXRs using the RANZCR reporting template. Overall, radiologists exhibit superior concordance in CXR categorization using the COVID-19 reporting template. Radiographers demonstrate wide variability, highlighting the need for enhanced education and training to standardise the triaging of these patients undergoing CXR imaging for COVID-19 symptoms.</div></div>","PeriodicalId":46420,"journal":{"name":"Journal of Medical Imaging and Radiation Sciences","volume":"56 5","pages":"Article 101911"},"PeriodicalIF":1.3000,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Medical Imaging and Radiation Sciences","FirstCategoryId":"1085","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S193986542500061X","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING","Score":null,"Total":0}

引用次数: 0

Abstract

Introduction

The Royal Australian and New Zealand College of Radiologists (RANZCR) has developed a reporting template to assist in the categorization of COVID-19 in chest X-ray (CXR) images and the levels of COVID-19 infection. Whilst CXRs are reported by radiologists, radiographers are often the first to assess the CXRs, and have the potential to support immediate triaging of patients with COVID-19. However, inter-reader concordance in the use of this reporting template remains underexplored.

Methods

70 CXR examinations comprising of the four categories in the RANZCR chest X-ray (CXR) COVID-19 reporting template were used for the study. These included: ‘typical’ (for COVID-19) (n = 30); ‘indeterminate’ (for COVID-19) (n = 20); ‘other diagnoses favoured’ (n = 10) and ‘normal’ (n = 10). These images were independently categorised using the RANZCR reporting template by three cohorts of readers: 12 radiologists, 13 registered radiographers, and 12 final-year radiography students. A Weighted Kappa (κ_w) was used to evaluate inter-reader agreement within and between the cohort of readers.

Results

Radiologists demonstrated fair (κ_w = 0.32) to substantial (κ_w = 0.77) inter-reader agreement, and their overall inter-reader was moderate (κ_w = 0.56). Registered radiographers demonstrated no (κ_w = -0.01) to moderate agreement (κ_w = 0.59), and their overall agreement was fair (κ_w = 0.31). Fourth year student radiographers demonstrated slight (κ_w = 0.004) to substantial (κ_w ₌ 0.8) agreement, with a moderate (κ_w = 0.47) overall agreement among final year student radiographers.

Conclusion

There are wide variations in the classification of the CXRs using the RANZCR reporting template. Overall, radiologists exhibit superior concordance in CXR categorization using the COVID-19 reporting template. Radiographers demonstrate wide variability, highlighting the need for enhanced education and training to standardise the triaging of these patients undergoing CXR imaging for COVID-19 symptoms.

查看原文本刊更多论文

利用RANZCR模板对COVID-19感染进行胸片图像分类的一致性研究

澳大利亚和新西兰皇家放射科医师学院（RANZCR）开发了一个报告模板，以协助对胸部x射线（CXR）图像中的COVID-19进行分类和COVID-19感染水平。虽然cxr由放射科医生报告，但放射科医生通常是第一个评估cxr的人，并有可能支持对COVID-19患者的即时分诊。然而，读者之间在使用该报告模板时的一致性仍未得到充分探讨。方法采用RANZCR胸片（CXR） COVID-19报告模板中4个类别的70例CXR检查进行研究。这些包括：“典型”（针对COVID-19）（n = 30）；“不确定”（COVID-19）（n = 20）；“其他诊断倾向”（n = 10）和“正常”（n = 10）。这些图像由三组读者使用RANZCR报告模板独立分类：12名放射科医生，13名注册放射技师和12名最后一年的放射学学生。加权Kappa （κw）用于评价读者群体内部和读者群体之间的读者间一致性。结果放射科医师的读者间一致性为一般（κw = 0.32）至相当（κw = 0.77），总体读者间一致性为中等（κw = 0.56）。注册放射技师没有（κw = -0.01）到中度一致（κw = 0.59），总体一致程度尚可（κw = 0.31）。四年级学生放射技师表现出轻微（κw = 0.004）到显著（κw = 0.8）的一致性，最后一年级学生放射技师表现出中度（κw = 0.47）的总体一致性。结论使用RANZCR报告模板对cxr的分类存在较大差异。总体而言，放射科医生在使用COVID-19报告模板进行CXR分类时表现出优越的一致性。放射技师表现出广泛的差异，这突出表明需要加强教育和培训，以标准化对这些接受CXR成像的COVID-19症状患者的分诊。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Medical Imaging and Radiation Sciences RADIOLOGY, NUCLEAR MEDICINE & MEDICAL IMAGING-

CiteScore

2.30

自引率

11.10%

发文量

231

审稿时长

53 days

期刊介绍： Journal of Medical Imaging and Radiation Sciences is the official peer-reviewed journal of the Canadian Association of Medical Radiation Technologists. This journal is published four times a year and is circulated to approximately 11,000 medical radiation technologists, libraries and radiology departments throughout Canada, the United States and overseas. The Journal publishes articles on recent research, new technology and techniques, professional practices, technologists viewpoints as well as relevant book reviews.