Improving Medical Student Leadership Evaluation Through Standard Setting​​​​​: A Comparison of Test Centered Versus Examinee Centered Methods of Standard Setting.

Abstract

Introduction: Physician trainees are increasingly expected to demonstrate leadership in complex, team-based clinical settings. However, assessing leadership performance in dynamic environments like simulation remains challenging, with limited consensus on valid, reliable standards. While some observational frameworks exist, few offer defensible methods for determining minimum passing thresholds, particularly in high-stakes contexts. Traditional test-centered approaches may not capture the behavioral complexity of leadership, whereas examinee-centered methods offer more context-sensitive evaluations. This study compares test-centered and examinee-centered standard-setting approaches to identify a fair, reproducible method for assessing leadership in simulation, offering a model that can be applied across medical education programs.

Methods: This study had 3 phases: first, the extended Angoff method (a test-centered method) was used to identify characteristics of borderline performers to set a cut-score; then we used the borderline group method (an examinee-centered method) to identify the cut-score; and finally, the cut-score was used to identify failing performance in a new data set.

Results: The expert panel in phase 1 determined 10.34 as the cut-score; therefore, a score of ≥11 would be required to pass the evaluation. Analyses conducted in phase 2 suggested that 13 should be the cut-score. The mean and SD for phase 3 were similar to those of phase 2, lending support for the cut score of 13.

Conclusions: Setting a cut-score is essential to accurately evaluating student performance when observations and ratings are more subjective. Ultimately, we sought to compare the test-centered versus examinee-centered methods and conclude that the examinee-centered method was a more holistic, fair, and reliable approach to student assessment.