Development and a validation study of comprehensive prescription writing rubrics for medical students

Abstract

Background: Prescribing is a complex task for physicians, with many global reports of errors. This study evaluates a comprehensive rubric for medical student prescribing skills regarding validity and reliability. Methods: Twenty-one third-year medical students participated in three separate prescribing exams. Two pharmacology professors rated the students' prescriptions using a rubric covering ten criteria. Messick validity framework was utilised to enhance the study’s validity. Generalisability theory (G-theory) helped determine the source of variance and the optimal number of raters and test occasions. Results: Content validity was ensured by three experts and alignment with the Thai Medical Council criterion. The Inter-rater and test-retest reliability were acceptable. The rubric had a Cronbach's alpha 0.70 with item-test correlation, all above 0.40. G-theory indicated that 54.93% of the total variance was due to performance and 27.57% to the interaction between performance and occasions, with a minimal residual variance of 4.28%. To reach an acceptable Phi-coefficient (≥0.70), three occasions with one rater (Phi-coefficient=0.76) or two occasions with two raters (Phi-coefficient=0.72) are needed. Conversely, the Phi-coefficient was low on a single occasion. Conclusion: The study introduces a comprehensive rubric and description of a prescription writing programme to minimise potential prescribing errors in pre-clinical years. Furthermore, more assessment opportunities enhance knowledge retention and assessment reliability.