The Allure of "Objective" Data in US Medical Student Assessment and Residency Selection-Are Faulty Data Better than None?

Abstract

Recent commentaries have urged reconsideration of the shift away from tiered grading in US medical schools and the removal of numerical scores on the US Medical Licensing Exam Step 1 examination. They suggest the move to pass/fail may negatively impact decision-making, especially affecting residency selection. By arguing for the need for data to help residency programs stratify applicants, they risk overstating the objectivity of "hard data" and overlooking the long-term goal of educating future doctors. We examine the history of tiered grading and explore the evidence that led to shifts away from such grading formats. Tiered systems often rely on multiple-choice question assessments that introduce bias and assess only medical knowledge. They show extreme variability and grade inflation, and studies have consistently demonstrated racial and gender disparities. Moreover, Step 1, which was never designed to stratify students' performances, has not been shown to predict future physician clinical success in any significant way. Claims that the shift away from tiered grading has resulted in detrimental student behaviors or bias against students from specific schools are unsubstantiated. In contrast, pass/fail grading leads to improved student well-being, intrinsic motivation, and mastery-oriented learning environments. Returning to competitive assessment, which stratifies residency applications, prioritizes a comfortable system over one with substantial validity evidence. Our efforts are better spent on developing new programmatic systems that incorporate holistic data, leverage artificial intelligence, and combine narrative feedback to assess our learners and prepare them to be competent physicians. We should strive to link assessments to long-term outcomes before using them to determine students' fates. We should not return to familiar data because they help residency programs stratify learners at the expense of their learning.