Reproducibility of the AGREE II Tool for Assessing the Methodological Quality of Clinical Practice Guidelines for the Management of Antithrombotic Agents in Patients Undergoing GI Endoscopy.

Background: Clinical practice guidelines (CPGs) exist for the management of antithrombotic agents in the periendoscopic period; however, their methodological qualities vary. The Appraisal of Guidelines for Research & Evaluation II (AGREE II) tool has been validated for the assessment of the methodological quality of CPGs; however, its reproducibility has not been assessed. The goal of this study was to assess the reproducibility of the AGREE II tool for CPGs published within the last 6 years for the management of antithrombotic agents in the periendoscopic period.

Study: A systematic search of PubMed and Embase databases was performed to identify eligible CPGs published between January 1, 2016 and April 14, 2022. The quality of the CPG was independently assessed by 6 reviewers using the AGREE II instrument. The reproducibility was summarized as weighted κ statistic and intraclass correlation coefficient using the SPSS statistical analysis package.

Results: The search yielded 343 citations with 7 CPGs from Europe, Asia, and the United States included in the critical appraisal. The overall mean weighted κ score across all guidelines was 0.300 (range, 0.093 to 0.384) indicating a fair agreement. The overall intraclass correlation coefficient was 0.462 (range, 0.175 to 0.570) for single measures and 0.837 (range, 0.560 to 0.888) for average measures indicating moderate reliability.

Conclusions: Our study shows only a fair overall interobserver agreement in the methodological quality of the included CPGs. The results suggest the need for education and training of CPG raters to enhance the application of the AGREE II tool to improve its reproducibility.