What tool do undergraduate pharmacy students prefer when grading systematic review evidence: AMSTAR-2 or ROBIS?

Cochrane Evidence Synthesis and Methods Pub Date : 2023-08-09 DOI:10.1002/cesm.12023

Shaun W. H. Lee

{"title":"What tool do undergraduate pharmacy students prefer when grading systematic review evidence: AMSTAR-2 or ROBIS?","authors":"Shaun W. H. Lee","doi":"10.1002/cesm.12023","DOIUrl":null,"url":null,"abstract":"<div>\n \n \n <section>\n \n <h3> Introduction</h3>\n \n <p>While systematic reviews (SRs) are considered the highest form of evidence in the hierarchy, the quality and standard of reviews varies. Two quality assessment tools have been developed to assess the variation in such standards. This study compared the preference, validity, reliability, and applicability of using A Measurement Tool to Assess Systematic Reviews (AMSTAR-2) and the Risk of Bias in Systematic Reviews (ROBIS) for critically appraising evidence by pharmacy students.</p>\n </section>\n \n <section>\n \n <h3> Materials and Methods</h3>\n \n <p>Students attended eight lectures on evidence-based medicine. Students independently assessed two SRs using AMSTAR-2 and ROBIS. The agreement between both tools were calculated using Spearman's test while interrater reliability was calculated using Fleiss' <i>κ</i> statistics.</p>\n </section>\n \n <section>\n \n <h3> Results</h3>\n \n <p>Students reported a preference for the AMSTAR-2 tool due to its clear and distinct rating criteria as well as guidance provided by the tool's developer. In comparison, students found the items on the ROBIS tool difficult to judge as it was subjective. A moderate agreement between both tools on the overall domain ratings was noted (Spearman <i>r</i><sub>s</sub> = 0.60). There was slight agreement in the overall confidence using AMSTAR-2 (<i>κ</i> = 0.05; 95% confidence interval [CI]: 0.01–0.12) and the overall domain in ROBIS (<i>κ</i> = 0.09; 95% CI: 0.01–0.16).</p>\n </section>\n \n <section>\n \n <h3> Conclusion</h3>\n \n <p>The AMSTAR-2 tool had a low level of concordance in ratings of review among students. However, the AMSTAR-2 tool was preferred by students due to the clear guidance and ease of use.</p>\n </section>\n </div>","PeriodicalId":100286,"journal":{"name":"Cochrane Evidence Synthesis and Methods","volume":"1 6","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2023-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1002/cesm.12023","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Cochrane Evidence Synthesis and Methods","FirstCategoryId":"1085","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1002/cesm.12023","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 0

Abstract

Introduction

While systematic reviews (SRs) are considered the highest form of evidence in the hierarchy, the quality and standard of reviews varies. Two quality assessment tools have been developed to assess the variation in such standards. This study compared the preference, validity, reliability, and applicability of using A Measurement Tool to Assess Systematic Reviews (AMSTAR-2) and the Risk of Bias in Systematic Reviews (ROBIS) for critically appraising evidence by pharmacy students.

Materials and Methods

Students attended eight lectures on evidence-based medicine. Students independently assessed two SRs using AMSTAR-2 and ROBIS. The agreement between both tools were calculated using Spearman's test while interrater reliability was calculated using Fleiss' κ statistics.

Results

Students reported a preference for the AMSTAR-2 tool due to its clear and distinct rating criteria as well as guidance provided by the tool's developer. In comparison, students found the items on the ROBIS tool difficult to judge as it was subjective. A moderate agreement between both tools on the overall domain ratings was noted (Spearman r_s = 0.60). There was slight agreement in the overall confidence using AMSTAR-2 (κ = 0.05; 95% confidence interval [CI]: 0.01–0.12) and the overall domain in ROBIS (κ = 0.09; 95% CI: 0.01–0.16).

Conclusion

The AMSTAR-2 tool had a low level of concordance in ratings of review among students. However, the AMSTAR-2 tool was preferred by students due to the clear guidance and ease of use.

Abstract Image

查看原文本刊更多论文

药学本科生在对系统审查证据进行评分时更喜欢什么工具：AMSTAR-2还是ROBIS？

引言虽然系统审查被认为是层次结构中的最高证据形式，但审查的质量和标准各不相同。已经开发了两个质量评估工具来评估这些标准的差异。本研究比较了使用评估系统评价的测量工具（AMSTAR-2）和系统评价中的偏倚风险（ROBIS）对药学学生批判性评价证据的偏好、有效性、可靠性和适用性。材料与方法学生参加了八场循证医学讲座。学生使用AMSTAR-2和ROBIS独立评估了两个SR。使用Spearman检验计算两种工具之间的一致性，而使用Fleissκ统计计算参与者间的可靠性。结果学生们报告说，他们更喜欢AMSTAR-2工具，因为它有明确而独特的评级标准以及工具开发人员提供的指导。相比之下，学生们发现ROBIS工具上的项目很难判断，因为它是主观的。两种工具在总体域名评级方面达成了适度一致（Spearman rs = 0.60）。使用AMSTAR-2（κ = 0.05；95%置信区间[CI]:0.01–0.12）和ROBIS中的整体域（κ = 0.09；95%可信区间：0.01–0.16）。结论AMSTAR-2工具在学生复习评分中的一致性水平较低。然而，AMSTAR-2工具由于其清晰的指导和易用性而受到学生的青睐。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Cochrane Evidence Synthesis and Methods

自引率

0.00%

发文量