AGREE II Quality Assessment of National and International Clinical Practice Guidelines on Prostate Cancer Management by the OPTIMA Consortium

Abstract

Background and objective

 Clinical practice guidelines for prostate cancer (PCa) are a valuable resource for everyday clinical practice. The clinical practice guidelines and recommendations produced by various societies should demonstrate a considerable level of consistency in terms of quality, regardless of the society that developed these given the common evidence base. However, to date, no study has assessed the quality of PCa clinical practice guidelines. As part of the Optimal Treatment for Patients with Solid Tumours in Europe Through Artificial intelligence (OPTIMA) project, we evaluated the quality of the most frequently used national and international clinical practice guidelines for PCa using the Appraisal of Guidelines for Research & Evaluation II (AGREE II) tool.

Methods

The quality of the identified clinical practice guidelines was assessed independently by two assessors using the AGREE II tool. The AGREE II tool comprises 23 different items organised into six domains, rated on a 7-point scale (1: strongly disagree to 7: strongly agree). The total score of the appraisal was the mean value of the two assessments. The agreement between assessors’ scores was calculated using the interclass correlation coefficient (ICC). Four key recommendations were compared among the included clinical practice guidelines to assess consistency.

Key findings and limitations

Sixteen clinical practice guidelines were assessed using their latest available version (cut-off April 2024). The European Association of Urology, S3LL PCa, Belgian Health Care Knowledge Centre, National Comprehensive Cancer Network, and Prostatacancer—Nationellt vårdprogram guidelines received the highest overall scores with a mean domain score of 82.4% (range: 75.5–88.3%). The de l’Association Française d’Urologie (AFU), American Urological Association, and National Institute for Health and Care Excellence received a mean domain score of 77.6% (range: 73.7–84.0%). Below average were the European Society for Medical Oncology, localised (L) and systemic (S) CPPC American Society of Clinical Oncology, and Nederlandse Vereniging voor Urologie (NVU) with a mean domain score of 58.4% (range: 43.5–76.3%). The reasons for scoring below average included the following: inadequate information about the methodology applied, limited scope of the guideline, and limited patient engagement. The highest inter-rater variability was observed in NVU (ICC: 0.58) and the lowest in AFU-L (ICC: 0.84). When examining the scores of each domain, “clarity of presentation” (domain 4) achieved the highest score with a mean of 86.9% ± 12.6%. The domain with the lowest score was applicability (domain 5), with a mean of 48.3% ± 24.8%. The ICC was calculated to be 0.72 (±0.08).

Conclusions and clinical implications

This is the first study in which a comprehensive quality assessment of the majority of international and national clinical practice guidelines was undertaken, and the key recommendations were compared to assess consistency. Our study shows that the majority of international and national clinical practice guidelines demonstrate high-quality standards when assessed using the AGREE II evaluation tool. The clinical practice guidelines that did not meet the expected standards could be improved by adopting several key recommendations outlined by our study.

Patient summary

The OPTIMA project used the Appraisal of Guidelines for Research & Evaluation II (AGREE II) tool to evaluate the quality of 16 commonly used national and international clinical practice guidelines for prostate cancer. While some of these international and national clinical practice guidelines received the highest score, few guidelines scored below average due to methodological deficiencies and limited patient engagement. These findings highlight the need for a standardised process to ensure high-quality, consistent guidelines across practices.