Evaluation of the reliability and usability of CARE-Radiology: A descriptive-analytic study

IF 3.6 2区医学 Q1 MEDICINE, GENERAL & INTERNAL

Journal of Evidence‐Based Medicine Pub Date : 2024-09-19 DOI:10.1111/jebm.12638

Mengshu Wang, Xufei Luo, Janne Estill, Karen Spruyt, Ryo Kurokawa, Nav Persaud, Yasuteru Shimamura, Holly Raison, Paolo Niccolò Franco, Cesare Maino, Hussein Elkhayat, Rehab A. Galal, Daisuke Kimura, Shingo Omata, Sergey A. Ryzhkin, Timur R. Izmailov, Rustem A. Bashirov, Zhaoxiang Bian, Jinhui Tian, Junqiang Lei

{"title":"Evaluation of the reliability and usability of CARE-Radiology: A descriptive-analytic study","authors":"Mengshu Wang, Xufei Luo, Janne Estill, Karen Spruyt, Ryo Kurokawa, Nav Persaud, Yasuteru Shimamura, Holly Raison, Paolo Niccolò Franco, Cesare Maino, Hussein Elkhayat, Rehab A. Galal, Daisuke Kimura, Shingo Omata, Sergey A. Ryzhkin, Timur R. Izmailov, Rustem A. Bashirov, Zhaoxiang Bian, Jinhui Tian, Junqiang Lei","doi":"10.1111/jebm.12638","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Aim</h3>\n \n <p>The study aimed to evaluate the reliability and usability of the CARE-Radiology checklist in assessing radiological case reports and provide a basis for its broader adoption and optimization.</p>\n </section>\n \n <section>\n \n <h3> Methods</h3>\n \n <p>Ten randomly selected radiological case reports published in scientific journals in 2020 were evaluated using the CARE-Radiology checklist. Twenty-six experts from 10 countries were invited to independently assess all ten reports. The reliability of the checklist was measured using Fleiss' Kappa, and Cronbach's alpha coefficient. Usability was evaluated by recording the time taken to complete the assessments and requesting the evaluators to rate each item on a Likert scale for its easiness of use.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>The median time for evaluating one radiological case report was 15 min. The overall agreement among evaluators showed moderate reliability with a Kappa value of 0.47 and a Cronbach's alpha of 0.51. The mean compliance rate for the items of CARE-Radiology was 61.8%, with some items exceeding 90% compliance. Items related to abstracts and keywords had the lowest compliance rates. The evaluators found most items easy to understand, with a few exceptions.</p>\n </section>\n \n <section>\n \n <h3> Conclusions</h3>\n \n <p>The CARE-Radiology checklist is relatively easy for researchers to use and understand. Continuous feedback is necessary for future revisions and updates, to enhance the effectiveness of the checklist, and to improve user experience.</p>\n </section>\n </div>","PeriodicalId":16090,"journal":{"name":"Journal of Evidence‐Based Medicine","volume":"17 3","pages":"597-603"},"PeriodicalIF":3.6000,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Evidence‐Based Medicine","FirstCategoryId":"3","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/jebm.12638","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MEDICINE, GENERAL & INTERNAL","Score":null,"Total":0}

引用次数: 0

Abstract

Aim

The study aimed to evaluate the reliability and usability of the CARE-Radiology checklist in assessing radiological case reports and provide a basis for its broader adoption and optimization.

Methods

Ten randomly selected radiological case reports published in scientific journals in 2020 were evaluated using the CARE-Radiology checklist. Twenty-six experts from 10 countries were invited to independently assess all ten reports. The reliability of the checklist was measured using Fleiss' Kappa, and Cronbach's alpha coefficient. Usability was evaluated by recording the time taken to complete the assessments and requesting the evaluators to rate each item on a Likert scale for its easiness of use.

Results

The median time for evaluating one radiological case report was 15 min. The overall agreement among evaluators showed moderate reliability with a Kappa value of 0.47 and a Cronbach's alpha of 0.51. The mean compliance rate for the items of CARE-Radiology was 61.8%, with some items exceeding 90% compliance. Items related to abstracts and keywords had the lowest compliance rates. The evaluators found most items easy to understand, with a few exceptions.

Conclusions

The CARE-Radiology checklist is relatively easy for researchers to use and understand. Continuous feedback is necessary for future revisions and updates, to enhance the effectiveness of the checklist, and to improve user experience.

查看原文本刊更多论文

评估 CARE-Radiology 的可靠性和可用性：描述性分析研究

研究旨在评估 CARE-Radiology 核对表在评估放射学病例报告中的可靠性和可用性，并为其更广泛的应用和优化提供依据。方法使用 CARE-Radiology 核对表评估随机选取的 2020 年发表在科学杂志上的十篇放射学病例报告。邀请来自 10 个国家的 26 位专家对所有 10 份报告进行独立评估。使用弗莱斯卡帕（Fleiss' Kappa）和克朗巴赫阿尔法系数（Cronbach's alpha coefficient）测量了核对表的可靠性。评估可用性的方法是记录完成评估所需的时间，并要求评估者用李克特量表对每个项目的易用性进行评分。评估者之间的总体一致性显示出中等可靠性，Kappa 值为 0.47，Cronbach's alpha 为 0.51。CARE-Radiology 项目的平均符合率为 61.8%，部分项目的符合率超过 90%。与摘要和关键词相关的项目符合率最低。结论 CARE-Radiology 核对表对于研究人员来说比较容易使用和理解。为提高核对表的有效性并改善用户体验，今后的修订和更新需要持续的反馈。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Evidence‐Based Medicine MEDICINE, GENERAL & INTERNAL-

CiteScore

11.20

自引率

1.40%

发文量

期刊介绍： The Journal of Evidence-Based Medicine (EMB) is an esteemed international healthcare and medical decision-making journal, dedicated to publishing groundbreaking research outcomes in evidence-based decision-making, research, practice, and education. Serving as the official English-language journal of the Cochrane China Centre and West China Hospital of Sichuan University, we eagerly welcome editorials, commentaries, and systematic reviews encompassing various topics such as clinical trials, policy, drug and patient safety, education, and knowledge translation.