Validation of the SYdney Shared and Evidence-based decision-Making test (SYStEM) among healthcare professionals

Abstract

Purpose

Whilst competence in evidence-based practice (EBP) is a critical skill for healthcare professionals, there is a gap in the literature in the assessment of EBP. Current frameworks in Evidence-Based Practice education cite a five-step process (Ask, Acquire, Appraise, Integrate and Evaluate) yet many existing measures of competence focus on the first three, relying on self-report and cognitive testing, falling short of assessing skills. The Fresno Test has been validated for assessing the first three steps and the “Assessing Communication about Patient Preferences” tool is reliable in capturing the quality of information given by physicians in integrating evidence. The Evidence-Based Practice Implementation Scale is useful for the ‘evaluate’ step in the EBP process. We adapted components from all three of these tools to create the SYdney Shared and Evidence-based decision-Making test (SYStEM) in order to measure EBP and Shared Decision Making (SDM) competencies in the context of patient care. The aims of this study were to examine the internal structure, criterion and consequential validity of the SYStEM.

Method

Data were available from 37 healthcare practitioners who completed the SYStEM online. We conducted a confirmatory factor analysis to determine the theoretical structure and a modified Angoff method to determine a minimum competence level for newly graduated health professionals.

Results

The final 10-item SYStEM consisted of two subscales: ‘EBP knowledge’ and ‘SDM skills’ demonstrating good construct validity and reliability (α > 0.74). The factorial validity of SYStEM was shown by all items loading significantly on their expected factors and a good fit to the data. Although the benchmark for passing the SYStEM was low, only 13/37 practitioners (35%) reached the minimum standard for EBP competence.

Discussion

This study provides preliminary evidence for the validity the SYStEM tool for measuring EBP competence amongst health graduates. Further large-scale validation of SYStEM is required.