Inter-rater reliability of the occupational violence risk assessment tool for emergency departments

Background

The three-item occupational violence (OV) risk assessment tool was developed and validated for use in emergency departments (EDs). It prompts review of each patient’s aggression history, behaviours, and clinical presentation. However, confidence around representativeness and generalisability are needed before widescale adoption; hence we measured the inter-rater reliability of the tool among a large group of emergency nurses.

Methods

A cross-sectional study was conducted between Sep 2021 and Jan 2022. Nurses were directed to a website that hosted an e-learning module about the tool. They were asked to apply the tool to two video scenarios of typical patient presentations. Demographic data, including years of emergency experience, were collected to contextualise their responses. Gwet’s Agreement Coefficients (AC1) were calculated to determine inter-rater reliability.

Results

There were 135 participants: typically female, under the age of 40 years, with more than 3 years of emergency nursing experience. Overall, there was excellent inter-rater agreement (AC1 =0.752, p = 0.001). This was consistent when years of ED experience was stratified: 0–2 years, AC1 = 0.764, p = 0.002; 3–5 years, AC1 = 0.826, p = 0.001; 6–10 years, AC1 = 0.751, p < 0.001; 11–15 years, AC1 = 0.659, p = 0.004; ≥ 16 years, AC1 = 0.799, p < 0.001.

Conclusion

The three-item OV risk assessment tool has excellent inter-rater reliability across a large sample of emergency nurses.