Qualitative evaluation of two web-based tools to improve accessibility of evidence reports

Abstract

Introduction

The Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) Program produces evidence reports to assist health systems in improving patient care. To make EPC evidence reports more accessible and usable, AHRQ convened a panel of learning health system (LHS) senior leaders to develop two web-based tools—a table-based tool presenting high-level results, and a graphical display that presented detailed data—to disseminate the reports and test the tools. Researchers examined (1) the context under which interview participants reviewed and used the evidence reported by the tools, (2) their experiences using the tools, (3) the tools' influence on clinical practice, and (4) how the tools could be improved.

Methods

Researchers collected and inductively analyzed qualitative data from tool implementation meetings with six LHSs and interviews with 27 LHS leaders and clinical staff who used the tools. Researchers used website utilization metrics to augment qualitative results.

Results

The tools were efficient, complementary, and useful sources of summarized evidence to promote system change, educate trainees and clinicians, inform research, and support shared decision making with patients and families. Clinical leaders appreciated the evidence review thoroughness and quality and viewed AHRQ as a trusted source of information. However, many felt the tools were not practical for bedside use because of their complex content. Participants also noted the reports had limited evidence strength and robustness. They suggested optimizing the tools for mobile device use to facilitate tool uptake and developing training resources about tool navigation and statistical content interpretation.

Conclusions

LHSs found the tools to be useful resources for making the EPC Program reports more accessible to and usable for health system leaders. The tools have the potential to meet some, but not all, LHS evidence needs. Their value depends on reports' usefulness, which ultimately depends on the evidence quality.