Intra-rater reliability, expertise and justifications for the American society of anesthesiologists physical status classification in dogs.

Abstract

Objective: The purposes of this study were to document intra-rater reliability in assigning American Society of Anesthesiologists Physical Status Classification System (ASA) status, to compare Cochran-Weiss-Shanteau (CWS) scores between anesthetists with varying levels of external validation, and to document the explanations given for assigning certain ASA statuses.

Study design: Prospective case series.

Subjects: A group of 29 anesthetists.

Methods: Anesthetists were presented with 25 hypothetical cases in two stages and asked to provide an ASA status as well as an explanation. Intra-rater reliability was calculated using the single-measure averaged intraclass correlation coefficient (ICC). The CWS index was calculated and quantifies expertise by assessing discrimination and consistency, where higher scores indicate superior judgment. Explanations for assignment of ASA status were analyzed by inductive coding by two investigators.

Results: The intra-rater ICC for all raters who completed Stage 1 and 2 was 0.80 (95% confidence interval 0.76-0.83, n = 17), 0.80 (0.74-0.86, n = 6) for Diplomates, 0.76 (0.68-0.82, n = 5) for residents and 0.86 (0.81-0.90, n = 6) for veterinary technicians. CWS ranged from 2.0 to 6.1, with the same median score of 3.1 for Diplomates, residents and technicians. In cases with the lowest agreement, the themes identified included disagreement on the degree of physiological compensation (e.g. tachycardia) and the effects of mild dehydration, an asymptomatic heart murmur, patients with brachycephalic airway syndrome and hypoproteinemia.

Conclusions and clinical relevance: Intra-rater reliability was high and similar regardless of level of external validation (i.e. technician vs. resident versus Diplomate). Using the CWS for ASA status assignment did not differentiate technicians, residents and Diplomates, although some individuals had relatively higher CWS, suggesting greater expertise in ASA status assignment. Disagreements exist in classifying conditions like brachycephalic airway syndrome and asymptomatic heart murmurs.