Evaluating Student Clerkship Performance Using Multiple Assessment Components

Abstract

Introduction: Family medicine clerkships utilize a broad set of objectives. The scope of these objectives cannot be measured by one assessment alone. Using multiple assessments aimed at measuring different objectives may provide more holistic evaluation of students. A further concern is to ensure longitudinal accuracy of assessments. In this study, we sought to better understand the relevance and validity of different assessment tools used in family medicine clerkships. Methods: We retrospectively correlated family medicine clerkship students’ scores across different assessments to evaluate the strengths of the correlations, between the different assessment tools. We defined ρ<0.3 as weak, ρ>0.3 to ρ<0.5 as moderate, and ρ>0.5 as high correlation. Results: We compared individual assessment scores for 267 students for analysis. The correlation of the clinical evaluation was 0.165 (P<.01); with case-based short-answer questions it was 0.153 (P<.01); and with objective structured clinical examinations it was -0.246 (P<0.01). Conclusion: Overall low levels of correlations between our assessments are expected, as they are each designed to measure different objectives. The relatively higher correlation between component scores supports convergent validity while correlations closer to zero suggest discriminant validity. Unexpectedly, comparing the multiple-choice questions and objective, structured clinical encounter (OSCE) assessments, we found higher correlation, although we believe these should measure disparate objectives. We replaced our in-house multiple-choice questions with a nationally-standardized exam and preliminary analysis shows the expected weaker correlation with the OSCE assessment, suggesting periodic correlations between assessments may be useful.