What makes a good guideline? - A systematic review and analysis of 120 clinical practice guidelines using the AGREE II tool.

Introduction: Clinical practice guidelines (CPGs) translate evidence into actionable recommendations to enhance care quality, improve clinical outcomes, reduce treatment variations, and make healthcare delivery more cost-effective. CPGs primarily aid healthcare practitioners but are also used by patients, policymakers, and organizations. The study aim was to assess CPG quality using the AGREE II instrument and to identify AGREE II items and domains that influence the overall assessment.

Methods: Medline and eight CPG databases were searched for guidelines applicable to older patients (≥60 years, frail, or with dementia/delirium) in acute orthopedic/traumatological settings, published in English or German since January 1st, 2016, and employing an evidence appraisal. Titles, abstracts, and full texts were independently screened by two reviewers using Covidence. AGREE II assessments were conducted across all 23 items in six domains by three reviewers with different professional backgrounds. Data were analyzed using descriptive statistics, Mann-Whitney U test, Pearson's r correlation matrix, variance inflation factor (VIF), univariable and multivariable regression (non-negative least squares), and intraclass correlation coefficient (ICC). Significance was set at p<0.05.

Results: A total of 120 CPGs have been appraised, reaching a mean overall rating of 4.35 (±1.13). Most guidelines received an overall rating of five (n=40, 33.33%), one guideline received an overall rating of one (0.8%). Using a standardized evidence rating framework (e.g. GRADE) is significantly associated with a better overall rating (p<0.001). The multivariable analysis showed that items 9, 12, and 15 had the highest influence on the overall AGREE II rating. Domain 6 (editorial independence) did not have an influence on the overall rating in a multivariable analysis.

Conclusion: Methodological rigor, particularly the use of a standardized evidence rating framework, is essential for a good overall AGREE II rating and thus for high-quality CPGs. The results from this analysis can assist different stakeholders who also conduct AGREE II appraisals, develop CPGs, or are charged with implementing CPG recommendations.