Reliability of a student patient education assessment rubric for continuous glucose monitor initiation

Objective

To validate a student patient education assessment rubric for initiation of a continuous glucose monitor (CGM) by measuring the inter-rater reliability (IRR).

Methods

Second and third-year pharmacy students at two institutions were evaluated on their ability to counsel a patient on the initial set up and use of a CGM device, either Dexcom G6 or FreeStyle Libre 2, during a practical assessment. Utilizing a standardized rubric, faculty evaluators evaluated each student. After course completion, three additional evaluators reviewed a recording of each session and evaluated the students utilizing the same rubric. The intraclass correlation coefficient [ICC(2,k)] was calculated to determine the IRR of the overall rubric and its four main sections: Introduction, Counseling Competency, Closing and Communication Ability.

Results

A total of 54 students completed the counseling practical assessment; one video recording was excluded for poor audio quality. Each student was graded using the standardized CGM rubric by four total faculty evaluators. The average student score was 23.1 out of 25 possible points. Median scores were slightly lower in students counseling on the Dexcom G6 device compared to Libre 2 (p = 0.005). The ICC(2,k) value among four evaluators was good [0.86, 95 % CI (0.71–0.93)], indicating a high level of agreement for the total rubric score.

Conclusion

A patient education assessment rubric for CGM device initiation demonstrated good inter-rater reliability across two institutions and may be a useful tool for institutions evaluating CGM counseling activities.