Use of tools for assessing the methodological quality of primary research in leading neurosurgical journals: A review of reviews

Abstract

Systematic reviews (SRs) and meta-analyses (MAs) require a comprehensive and reproducible strategy to assess the methodological quality of the included studies. This research-on-research study evaluated the methods used to assess research quality by SRs and MAs published in leading neurosurgical journals, and identified factors associated with the publication of a comprehensive and reproducible assessment. We systematically surveyed SRs published in the 10 leading neurosurgical journals between 01/11/2019 and 31/12/2021. PubMed was used to search the MEDLINE database, which was supplemented by individual journal searches. Included SRs were assessed using a standardised data extraction tool. Descriptive statistics were utilised to identify factors associated with methodological and reporting quality of the tool-based quality assessment. A total of 564 SRs were included in the analysis. 326 (57.80%) included MAs, 165 (29.26%) included at least one Randomized Controlled Trial (RCT) and 29 (5.14%) included only RCTs. Scales were the most commonly used tool for methodological quality assessment (32.45%), followed by domain-based tools (24.82%) and checklists (9.93%). The number of included studies was inversely associated with multiple methodological quality assessment metrics. A positive association was observed between the number of included patients and multiple methodological quality assessment metrics. We established that the methodological and reporting quality of tool-based quality assessment requires improvement. This issue is particularly pertinent for SRs limited to non-randomised studies, which account for the vast majority of neurosurgical SRs. We recommend the use of domain-based tools for methodological quality assessment as these provide a more nuanced assessment of methodological quality.