Penicillin allergy risk stratification tool for use by non-allergists in the peri-operative setting: an agreement study and qualitative process evaluation.

INTRODUCTION Many patients are labelled incorrectly as 'penicillin allergic'. This label is associated with poor health outcomes, regardless of whether the allergy is genuine. We designed a study to assess agreement between allergy specialists (consultant allergists or immunologists) and non-allergists when using a risk stratification tool to determine suitability for a penicillin challenge test. METHODS In this two-centre mixed-method study, adult patients undergoing surgery who had been labelled penicillin allergic were assessed by a healthcare professional with no formal allergy training (consultant anaesthetist or peri-operative nurse) and an allergy specialist doctor using the risk stratification tool. Cohen's kappa was used to quantify agreement. We conducted a qualitative process evaluation based on individual and group interviews to explore user experience. RESULTS We recruited 139 patients and 11 healthcare professionals; 101 patients completed both assessments (median (IQR [range]) age was 53 (40-66 [19-86]) y, 71% women). Allergist and non-allergist assessments agreed for 63 (62%) patients. In the remaining 38 (38%), most disagreements represented an overcautious approach by non-allergists, except in seven patients categorised as unsuitable (or uncertain) by the allergist but suitable by the non-allergist. Inter-rater agreement was moderate (unweighted κ 0.43, 95%CI 0.30-0.57; weighted κ 0.47, 95%CI 0.27-0.67). The prevalence and bias-adjusted score indicated poor inter-rater reliability (κ score 0.25, 95%CI 0.06-0.44). Qualitative analysis indicated low confidence in the tool among non-allergists, with a perceived need for allergy training and specialist support. DISCUSSION This penicillin allergy risk stratification tool, designed for use by healthcare professionals with no formal allergy training, showed poor agreement with allergy expert assessments.