The impact of ethnicity information in single best answer questions on student performance, item response time and psychometric properties

Abstract

Inclusion of patient age and sex in single best answer (SBA) questions is standard practice, whereas ethnicity is typically included only when considered clinically relevant. This selective use may inadvertently cue students or introduce bias. This study examined whether including patient ethnicity in SBA questions influences student performance, response time and item psychometric properties. A prospective randomised controlled study was conducted using two versions of a final-year formative assessment. In each version, approximately half of the questions included the patient’s ethnicity, while the corresponding questions in the alternate version omitted this information. Students were randomly assigned to one of the two exam versions. Two hundred and sixty-seven final-year medical students participated in the study. The mean score for questions without ethnicity was 80.3% and for questions with ethnicity was 79.8% (mean difference = −0.5 pp, p = 0.12). On average, students spent 1.6 seconds longer to answer questions with ethnicity compared to questions without ethnicity (p < 0.01). In SBA questions with and without ethnicity, there was no significant difference in mean facility (79.8% vs. 80.3%, p = 0.17) or mean item–total score point-biserial correlation (0.20 vs. 0.21, p = 0.63). These indicate that including patient ethnicity did not significantly impact overall student performance or mean item-level psychometric properties. However, it was associated with longer response times, suggesting a small but significant increase in cognitive load. Educators involved in assessment design should carefully consider this added cognitive burden when deciding whether to include patient ethnicity in future assessments.