Determining the methodological rigor and overall quality of out-of-hospital clinical practice guidelines: a scoping review.

Abstract

Objectives: Out-of-hospital clinical practice guidelines (CPGs) guide paramedics, emergency medical technicians and first responders, but their quality remains uncertain. This scoping review aims to identify, aggregate and describe all literature that has used a structured appraisal instrument to assess the methodological rigor and overall quality of out-of-hospital CPGs.

Methods: This study was conducted in accordance with the JBI methodology for scoping reviews and involved systematically searching the following databases and/or information sources with no publication or language limit applied: MEDLINE (Ovid), Embase (Elsevier), CINAHL with full text (EBSCO), Scopus (Elsevier), ProQuest Central (ProQuest).

Results: This review identified 15 articles that appraised 311 unique out-of-hospital CPGs. These CPGs ranged in date of publication from 1998 to 2022. The majority of CPGs (267/311) were assessed using the Appraisal of Guidelines for Research & Evaluation (AGREE-II) instrument, with 146 guidelines appraised against two tools. Following aggregation, CPGs scored highest in Domain 4 (clarity of presentation) at 77.7% (SD = 15.1%), and lowest in Domain 5 (applicability) at 42.6% (SD = 23.7%). The average Domain 3 score (rigor of development) was 55.6% (SD = 25.7%). Of CPGs appraised against the AGREE-II instrument, 34.4% met our a priori definition of being high-quality (Domain 3 score of equal to or greater than 75%), while 31.3% were deemed medium-quality (Domain 3 score between 74% and 50%), and 34.3% were considered low-quality (Domain 3 score less than 50%). There were no significant changes observed in the average Domain 3 score over time (p = 0.092). 146 CPGs were assessed against the National Academy of Medicine criteria with 34.9% meeting all elements indicative of being a high-quality guideline, while 39 CPGs were assessed the 2016 National Health and Medical Research Council Standards for Guidelines with 0% meeting all criteria.

Conclusions: Out-of-hospital CPGs currently have poor methodological rigor and are of medium to low overall quality. These results should be used to inform future research and initiatives that aim to standardize the methods used to develop guidelines used in this healthcare setting.