Echocardiographic Diagnosis of Rheumatic Heart Disease: Substantial Agreement, Inter-Rater Reliability and Accuracy Between Cardiac Sonographers and Paediatric Cardiologists

IF 2.2 4区医学 Q2 CARDIAC & CARDIOVASCULAR SYSTEMS

Heart, Lung and Circulation Pub Date : 2025-09-01 DOI:10.1016/j.hlc.2025.03.001

Jacqueline M. Williamson BSc , Gillian A. Whalley PhD , Peter S. Morris MBBS, PhD , Dianne Bruce BSc , Michael Cursaro BMS , Ari Horton MBBS , Robert Justo MBBS , Catherine H. Mylrea GradDipCardiacUltrasound , Suzanne Davy-Snow BSc , Sela Takau BASc , Gavin R. Wheaton MBBS , Amy N. Zanker BSc , Zhiqiang Wang PhD , Bo Remenyi MBBS, PhD

{"title":"Echocardiographic Diagnosis of Rheumatic Heart Disease: Substantial Agreement, Inter-Rater Reliability and Accuracy Between Cardiac Sonographers and Paediatric Cardiologists","authors":"Jacqueline M. Williamson BSc , Gillian A. Whalley PhD , Peter S. Morris MBBS, PhD , Dianne Bruce BSc , Michael Cursaro BMS , Ari Horton MBBS , Robert Justo MBBS , Catherine H. Mylrea GradDipCardiacUltrasound , Suzanne Davy-Snow BSc , Sela Takau BASc , Gavin R. Wheaton MBBS , Amy N. Zanker BSc , Zhiqiang Wang PhD , Bo Remenyi MBBS, PhD","doi":"10.1016/j.hlc.2025.03.001","DOIUrl":null,"url":null,"abstract":"<div><h3>Background</h3><div>Rheumatic heart disease (RHD) rates remain high in many socially disadvantaged populations. Early detection through active case finding is resource intensive and task-sharing with cardiac sonographers may improve access to screening. The accuracy of cardiac sonographers in the echocardiographic evaluation of RHD has not previously been assessed. The 2012 World Heart Federation guidelines were used to compare agreement, inter-rater reliability, and accuracy between cardiac sonographers and cardiologists in the echocardiographic evaluation of RHD.</div></div><div><h3>Method</h3><div>Four paediatric cardiologists and seven cardiac sonographers underwent structured training before independently reporting the same 210 echocardiograms (a total of 2,310 individual studies). Participants were blind to the results of other assessors and blind to the original diagnoses by a three-person expert panel consensus. Median results from each group of professionals were compared to each other and the original diagnosis. Individual results were also analysed to determine agreement with the original panel diagnosis using Cohen’s kappa. Inter-rater agreement for multiple assessors in each group was determined using Fleiss’ kappa. Sensitivity, specificity, and likelihood ratios were estimated using the original panel diagnosis as the reference standard.</div></div><div><h3>Results</h3><div>The agreement was substantial between median cardiac sonographer and median cardiologist results for the detection of any abnormality (<em>k</em>=0.74, 95% CI 0.61–0.88); any RHD (<em>k</em>=0.79, 95% CI 0.65–0.92); and definite RHD (<em>k</em>=0.78, 95% CI 0.64–0.91). Accuracy for the detection of any RHD was high with sensitivity of median cardiologist results of 79% (95% CI 71%–86%) and specificity of 97% (95% CI 87%–99%) and median sonographer sensitivity was 83% (95% CI 75%–89%) with specificity of 93% (95% CI 85%–98%).</div></div><div><h3>Conclusions</h3><div>Expert cardiac sonographers achieved a similar level of agreement, inter-rater reliability, and accuracy to expert cardiologists in the reporting of RHD. These findings support the use of task-sharing for the echocardiographic detection of RHD.</div></div>","PeriodicalId":13000,"journal":{"name":"Heart, Lung and Circulation","volume":"34 9","pages":"Pages 900-907"},"PeriodicalIF":2.2000,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Heart, Lung and Circulation","FirstCategoryId":"3","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1443950625001738","RegionNum":4,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"CARDIAC & CARDIOVASCULAR SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

Background

Rheumatic heart disease (RHD) rates remain high in many socially disadvantaged populations. Early detection through active case finding is resource intensive and task-sharing with cardiac sonographers may improve access to screening. The accuracy of cardiac sonographers in the echocardiographic evaluation of RHD has not previously been assessed. The 2012 World Heart Federation guidelines were used to compare agreement, inter-rater reliability, and accuracy between cardiac sonographers and cardiologists in the echocardiographic evaluation of RHD.

Method

Four paediatric cardiologists and seven cardiac sonographers underwent structured training before independently reporting the same 210 echocardiograms (a total of 2,310 individual studies). Participants were blind to the results of other assessors and blind to the original diagnoses by a three-person expert panel consensus. Median results from each group of professionals were compared to each other and the original diagnosis. Individual results were also analysed to determine agreement with the original panel diagnosis using Cohen’s kappa. Inter-rater agreement for multiple assessors in each group was determined using Fleiss’ kappa. Sensitivity, specificity, and likelihood ratios were estimated using the original panel diagnosis as the reference standard.

Results

The agreement was substantial between median cardiac sonographer and median cardiologist results for the detection of any abnormality (k=0.74, 95% CI 0.61–0.88); any RHD (k=0.79, 95% CI 0.65–0.92); and definite RHD (k=0.78, 95% CI 0.64–0.91). Accuracy for the detection of any RHD was high with sensitivity of median cardiologist results of 79% (95% CI 71%–86%) and specificity of 97% (95% CI 87%–99%) and median sonographer sensitivity was 83% (95% CI 75%–89%) with specificity of 93% (95% CI 85%–98%).

Conclusions

Expert cardiac sonographers achieved a similar level of agreement, inter-rater reliability, and accuracy to expert cardiologists in the reporting of RHD. These findings support the use of task-sharing for the echocardiographic detection of RHD.

查看原文本刊更多论文

风湿性心脏病的超声心动图诊断：心脏超声医师和儿科心脏病专家之间的实质性一致、可靠性和准确性。

背景：风湿性心脏病（RHD）的发病率在许多社会弱势人群中仍然很高。通过主动病例发现的早期检测是资源密集型的，与心脏超声医师共享任务可能会改善筛查的可及性。心脏超声心动图在RHD超声心动图评价中的准确性以前没有被评估过。2012年世界心脏联合会指南用于比较超声心动图评估RHD时心脏超声医师和心脏病专家之间的一致性、可靠性和准确性。方法：四名儿科心脏病专家和七名心脏超声技师在独立报告相同的210张超声心动图（总共2310项个体研究）之前接受了结构化培训。参与者不知道其他评估者的结果，也不知道由三人专家小组一致做出的原始诊断。每组专业人员的中位数结果相互比较并与原始诊断进行比较。还分析了个体结果，以确定是否与使用科恩卡帕的原始小组诊断一致。使用Fleiss kappa确定每组中多个评估者的评分间一致性。敏感度、特异性和似然比使用原始的小组诊断作为参考标准进行估计。结果：心脏超声检查的中位结果与心脏病专家的中位结果在检测任何异常方面的一致性是显著的（k=0.74, 95% CI 0.61-0.88）；任何RHD (k=0.79, 95% CI 0.65-0.92)；明确RHD （k=0.78, 95% CI 0.64-0.91）。任何RHD检测的准确性都很高，心脏病专家结果的中位灵敏度为79% (95% CI 71%-86%)，特异性为97% (95% CI 87%-99%)，超声检查结果的中位灵敏度为83% (95% CI 75%-89%)，特异性为93% （95% CI 85%-98%）。结论：专家心脏超声医师在报告RHD时达到了与专家心脏病专家相似的一致性、可靠性和准确性。这些发现支持在RHD超声心动图检测中使用任务共享。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Heart, Lung and Circulation CARDIAC & CARDIOVASCULAR SYSTEMS-

CiteScore

4.50

自引率

3.80%

发文量

912

审稿时长

11.9 weeks

期刊介绍： Heart, Lung and Circulation publishes articles integrating clinical and research activities in the fields of basic cardiovascular science, clinical cardiology and cardiac surgery, with a focus on emerging issues in cardiovascular disease. The journal promotes multidisciplinary dialogue between cardiologists, cardiothoracic surgeons, cardio-pulmonary physicians and cardiovascular scientists.