Reliability and Validity of the New Urgency Classification Model for Acute Medical Care in Dutch Nursing Home Setting

Abstract

Objective

To validate an Urgency Classification Model developed for telephone triage in Dutch nursing homes.

Design

Retrospective observational study.

Setting and Participants

Retrospective analysis of triage data of nursing home residents in a medical service organization, active in 40 nursing homes across the Netherlands.

Methods

An Urgency Classification Model for nursing home care was developed through a collaborative cocreation session by modifying existing acute medical care delivery models. All inquiries to central triage personnel during regular working hours between April 1 and April 30, 2022, were retrospectively categorized according to the new Urgency Classification Model (“urgency,” consisting of 6 levels from U0 to U5; and “goals of care and treatment limitations,” consisting of 4 options) by 2 independent physicians to evaluate the reliability using Cohen's kappa. To ascertain validity, the categorized data were juxtaposed with the executed treatment plan as documented in the patient records.

Results

Of 387 inquiries, consensus between assessors using the Urgency Classification Model was reached upon initial independent classification of urgency in 77.0% (n = 298, Cohen's kappa 0.654) of cases and in 77.3% (n = 299, Cohen's kappa 0.649) of goals of care and treatment limitations classification, representing substantial interrater reliability. A strong positive correlation was found between the urgency identified through the Urgency Classification Model and the observed urgency in the executed treatment, r_s = 0.662, P < .001; the same urgency was given in 71.5% (n = 276) of all inquiries. Overtriage (meaning the model classified the inquiry as more urgent than the executed treatment plan) occurred in 9.8% (n = 38) and undertriage in 18.7% (n = 72).

Conclusion and Implications

The new Urgency Classification Model is a valid and reliable classification tool for implementation within its intended target population. Universal and comprehensive implementation is expected to lead to more appropriate care delivery, while realizing integration with the acute medical care frameworks already in place.