Feasibility and Reliability of the Adapted Kagan Scales for Rating Conversations for People With Acquired Brain Injury: A Multiphase Iterative Mixed-Methods Design.

Abstract

Purpose: Rating the quality of conversations can assess communication skills in both people with acquired brain injury and their communication partners. This study explored the clinical feasibility and reliability of two conversation rating scales: the Adapted Measure of Participation in Conversation (MPC) and the Adapted Measure of Support in Conversation (MSC).

Method: Raters were final-year speech and language therapy students (n = 14) and qualified clinicians (n = 2). Raters attended training on the Adapted MPC and MSC, watched 5 or 10 min of videotaped conversations (n = 23), and then scored them on the MPC and MSC scales. Data were collected over four phases, which varied according to the length of the training, sample length, number of samples rated, and level of clinical expertise. Feasibility data (time taken to score conversations and ease of use) were collected. Interrater reliability was assessed using intraclass correlations (ICCs: absolute agreement, single measures).

Results: Raters took 30-45 min to score a 10-min sample, and they took 20-30 min to score a 5-min sample. Ease of use was rated highly across all phases. Overall reliability for rating 5 min of conversation (ICC = .52-.73) was better than for 10 min of conversation (ICC = .33-.68). Reliability for the MPC was moderate for both students (ICC = .69) and clinicians (ICC = .55), and for the MSC, it was moderate for both students (ICC = .73) and clinicians (ICC = .58). Reliability was better for students compared with clinicians.

Conclusions: Rating a 5-min conversation in under 30 min was feasible, with more reliable results for 5-min compared with 10-min conversations. Implications for assessing conversation in the future are discussed.