Inter-rater Reliability of the Reflux Sign Assessment-10 (RSA-10).

Abstract

Objective: To evaluate the inter-rater reliability and internal consistency of the Reflux Sign Assessment-10 (RSA-10) among otolaryngologists and speech therapists with various experiences.

Methods: Six experts (2 otolaryngologists, 2 speech-therapists, and 2 speech-therapist students) rated 300 clinical images of oral, laryngeal, and pharyngeal signs from patients with laryngopharyngeal reflux disease diagnosis at the 24-hour hypopharyngeal-esophageal multichannel intraluminal impedance-pH monitoring. Inter-rater reliability and internal consistency were evaluated with Intraclass Correlation (ICC) and Cronbach-α. The severity of scores was compared between judges. The intra-rater (test-retest) reliability was evaluated with the Spearman correlation coefficient.

Results: The pictures of 40 patients were included. There were 18 females and 22 males. The mean age was 52.6 ± 13.9 years. The Cronbach-α was 0.854, which indicates a high internal consistency between judges. The overall ICC was 0.787 (95% CI: 0.715-0.845; P = 0.001). The ICC varied among judges with the highest value for students (ICC = 0.960) and SLP seniors versus students (ICC = 0.805). The severity of RSA-10 rating scores was influenced by the number of reflux patients seen (r_s =-0.941; P = 0.001) and the number of fiberscope examinations performed (r_s =-0.812; P = 0.049). The RSA-10 was more severely scored by speech therapists with the least experience compared to otolaryngologists with the most experience in fiberscope/reflux patient assessment.

Conclusion: The RSA-10 demonstrated adequate global ICC and internal consistency among otolaryngologists and speech therapists with various degrees of experience. The assessment of RSA was influenced by the fibroscopy experience, and the number of reflux patients seen.