Serious Adverse Events: A Replicability and Validation Study of Naranjo Causality Assessment Tool in a Canadian Clinical Setting

Rationale

Patient safety has become a major concern in healthcare today as 5%–10% of patients experience serious adverse events (SAE) during their hospital stay. The causal assessment of SAE is the responsibility of healthcare professionals (HCP), who use their judgment or a standardize tool. Whether those two methods are replicable to provide similar results remains unclear.

Objective

Our aim was to evaluate if causality assessment performed by HCP is replicable when systematically assessed with the Naranjo tool and to validate its performance in Canadian clinical context.

Methods

We performed pilot retrospective cohort study which included patients with SAE admitted to a Quebec hospital in 2021. Twelve SAE were randomly selected, and two reviewers independently assessed their causality using Naranjo tool. Inter-rater reliability among two reviewers and between HCP was evaluated. Along with criterion validity, sensitivity and specificity were calculated for validation study.

Results

Weighted kappa was 0.92 (good inter-rater reliability) where kappa was 0.84 (good agreement between reviewers). No causality assessment by HCP was documented leading to impossibility in computing replicability. The Naranjo tool showed positive monotonic correlation with expert opinion resulting in r_s = 0.208 (p < 0.001). Classification of Naranjo scores to binary variables resulted in sensitivity of 1.00 and specificity of 0.31.

Conclusion

Our study suggested that Naranjo tool is reliable and valid to be used in a clinical setting and was able to classify all drug products involved in the occurrence of SAE. Larger scale studies need to be conducted in real-time clinical settings to investigate its performance and utility.