Methodologic quality and risk-of-bias in systematic reviews of healthcare interventions: a review of methods

Q3 Medicine

International Journal of Evidence-Based Healthcare Pub Date : 2022-05-18 DOI:10.17267/2675-021xevidence.2022.e4067

Cristine Miron Stefani, Liliana Vicente Melo de Lucas Rezende, Lia Rosana Honnef, L. Butini Oliveira, Nathan Da Cruz Lopes, C. Massignan, G. De Luca Canto, Júlia Meller Dias de Oliveira

{"title":"Methodologic quality and risk-of-bias in systematic reviews of healthcare interventions: a review of methods","authors":"Cristine Miron Stefani, Liliana Vicente Melo de Lucas Rezende, Lia Rosana Honnef, L. Butini Oliveira, Nathan Da Cruz Lopes, C. Massignan, G. De Luca Canto, Júlia Meller Dias de Oliveira","doi":"10.17267/2675-021xevidence.2022.e4067","DOIUrl":null,"url":null,"abstract":"OBJECTIVE: To compare the characteristics of systematic reviews of healthcare interventions that assessed or did not methodologic quality/risk-of-bias of included studies. Additionally, to analyze methodologic features of those assessing the methodologic quality/risk-of-bias. METHODS: PubMed database was searched. From 25,571 systematic reviews retrieved, a random sample of 1,025 was screened. Frequencies were used to describe outcomes. Unadjusted and adjusted logistic regressions were performed to test the associations with methodologic quality/risk-of-bias results assessment. In a second analysis, systematic reviews that assessed methodologic quality/risk of bias were dichotomized according to the design of included studies (randomized clinical trials-only versus non-randomized studies of intervention or a combination of both). RESULTS: 303 systematic reviews were fully analyzed. Methodologic quality/risk of bias was assessed by 278 (92%). Methodologic quality/risk-of-bias assessment was associated with a higher number of databases searched (>4, P= 0.008), the presence of meta-analysis (P= 0.005), and the design of included studies (randomized clinical trials-only, P= 0.042). The chance of using a suitable tool and a tool designed for risk-of-bias assessment rather than methodologic quality was higher for randomized clinical trials-only systematic reviews (P< 0.05). The most used tool was Cochrane’s RoB Tool without a clear studies’ overall risk classification system. CONCLUSION: methodologic quality/risk-of-bias assessment was associated with included studies’ design (randomized clinical trials-only), a meta-analysis of data, and the number of databases searched (>4). The most used tool was Cochrane’s RoB Tool, with no clearly defined rating system. Methodologic quality/risk-of-bias assessment methods description, results, and impacts on meta-analysis, the certainty of evidence, and systematic reviews’ results are still to be consistently addressed.\n ","PeriodicalId":55996,"journal":{"name":"International Journal of Evidence-Based Healthcare","volume":"25 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2022-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Evidence-Based Healthcare","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.17267/2675-021xevidence.2022.e4067","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"Medicine","Score":null,"Total":0}

引用次数: 0

Abstract

OBJECTIVE: To compare the characteristics of systematic reviews of healthcare interventions that assessed or did not methodologic quality/risk-of-bias of included studies. Additionally, to analyze methodologic features of those assessing the methodologic quality/risk-of-bias. METHODS: PubMed database was searched. From 25,571 systematic reviews retrieved, a random sample of 1,025 was screened. Frequencies were used to describe outcomes. Unadjusted and adjusted logistic regressions were performed to test the associations with methodologic quality/risk-of-bias results assessment. In a second analysis, systematic reviews that assessed methodologic quality/risk of bias were dichotomized according to the design of included studies (randomized clinical trials-only versus non-randomized studies of intervention or a combination of both). RESULTS: 303 systematic reviews were fully analyzed. Methodologic quality/risk of bias was assessed by 278 (92%). Methodologic quality/risk-of-bias assessment was associated with a higher number of databases searched (>4, P= 0.008), the presence of meta-analysis (P= 0.005), and the design of included studies (randomized clinical trials-only, P= 0.042). The chance of using a suitable tool and a tool designed for risk-of-bias assessment rather than methodologic quality was higher for randomized clinical trials-only systematic reviews (P< 0.05). The most used tool was Cochrane’s RoB Tool without a clear studies’ overall risk classification system. CONCLUSION: methodologic quality/risk-of-bias assessment was associated with included studies’ design (randomized clinical trials-only), a meta-analysis of data, and the number of databases searched (>4). The most used tool was Cochrane’s RoB Tool, with no clearly defined rating system. Methodologic quality/risk-of-bias assessment methods description, results, and impacts on meta-analysis, the certainty of evidence, and systematic reviews’ results are still to be consistently addressed.

查看原文本刊更多论文

卫生保健干预系统评价的方法学质量和偏倚风险:方法综述

目的:比较评估或不评估纳入研究的方法学质量/偏倚风险的医疗干预措施的系统评价的特点。此外，分析评估方法学质量/偏倚风险的方法学特征。方法:检索PubMed数据库。从25,571篇系统综述中，随机筛选1,025份样本。频率用于描述结果。采用未调整和调整的逻辑回归来检验与方法学质量/偏倚风险结果评估的相关性。在第二项分析中，评估方法学质量/偏倚风险的系统综述根据纳入研究的设计(仅随机临床试验与干预的非随机研究或两者的结合)进行了分类。结果:对303篇系统评价进行了全面分析。评估方法学质量/偏倚风险的有278人(92%)。方法学质量/偏倚风险评估与较高的数据库检索数量(>4,P= 0.008)、meta分析的存在(P= 0.005)和纳入研究的设计(仅随机临床试验，P= 0.042)相关。仅随机临床试验的系统评价使用合适的工具和专为偏倚风险评估而设计的工具的机会高于方法学质量(P< 0.05)。使用最多的工具是Cochrane的RoB工具，但没有明确的研究总体风险分类系统。结论:方法学质量/偏倚风险评估与纳入研究的设计(仅随机临床试验)、数据的荟萃分析和检索数据库的数量(>4)相关。最常用的工具是Cochrane的RoB工具，没有明确定义的评级系统。方法学质量/偏倚风险评估方法的描述、结果和对meta分析的影响、证据的确定性和系统评价结果仍有待一致解决。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

International Journal of Evidence-Based Healthcare Medicine-Health Policy

CiteScore

1.80

自引率

0.00%

发文量

期刊介绍： The International Journal of Evidence-Based Healthcare is the official journal of the Joanna Briggs Institute. It is a fully refereed journal that publishes manuscripts relating to evidence-based medicine and evidence-based practice. It publishes papers containing reliable evidence to assist health professionals in their evaluation and decision-making, and to inform health professionals, students and researchers of outcomes, debates and developments in evidence-based medicine and healthcare. The journal provides a unique home for publication of systematic reviews (quantitative, qualitative, mixed methods, economic, scoping and prevalence) and implementation projects including the synthesis, transfer and utilisation of evidence in clinical practice. Original scholarly work relating to the synthesis (translation science), transfer (distribution) and utilization (implementation science and evaluation) of evidence to inform multidisciplinary healthcare practice is considered for publication. The journal also publishes original scholarly commentary pieces relating to the generation and synthesis of evidence for practice and quality improvement, the use and evaluation of evidence in practice, and the process of conducting systematic reviews (methodology) which covers quantitative, qualitative, mixed methods, economic, scoping and prevalence methods. In addition, the journal’s content includes implementation projects including the transfer and utilisation of evidence in clinical practice as well as providing a forum for the debate of issues surrounding evidence-based healthcare.