Educational Download: Examinations Over Time by the Numbers

Abstract

Standardized examinations such as the United States Medical Licensing Examination (USMLE) and American Board of Emergency Medicine Qualifying Examination (ABEM-QE), the written examinations, serve as measures of medical knowledge and application of knowledge to patient care. These are intended to serve as gateways to licensure and practice and thus serve as key outcomes for training in Emergency Medicine (EM).

Flynn observed that over the past decades, there has been an increase in scores on some standardized tests, such as the Intelligence Quotient (IQ) tests [1]. Similarly, USMLE Step 2 scores have been rising [2]. Starting before COVID and amplified during COVID, EM has seen a decrease in scores on EM-specific first-time examinations, which is concerning [3].

The purpose of this Educational Download is to provide a snapshot of examination scores over the continuum of training. Thus, we follow a cohort of trainees from USMLE Steps 1 and 2 (and COMLEX 1 and 2) [4], through In-Training Examination (ITE) scores to board qualification (the written ABEM-QE). We followed the index cohort through their series of examinations (assuming a 3-year residency program).

Since the change of Step 1 to pass/fail, pass rates have decreased while Step 2 pass rates and scores are high. Importantly, there are decreases in EM ITE and ABEM-QE scores and pass rates during and since COVID.

One caveat is that the majority of these examinations are equated, a process by which scores from one test may be weighted so as to have equal meaning with scores from another test. Another limitation is that some examinations report on the calendar year (December–January) while others report on the academic year (July–June); exact alignment of the scores for cohorts is not possible versus scaled scores. Scoring of examinations is imprecise, and the fluctuations in scores could be within the standard error of measurement [6]. Finally, some examinations have equating changes or standard settings related cut score changes that are not reflected here.

Sally A. Santen: conceptualization; writing – original draft; visualization; writing – review and editing. Jessica Baez: conceptualization; writing – original draft. Susan Promes: conceptualization; writing – original draft; writing – review and editing. Anne Messman: writing – original draft; conceptualization; writing – review and editing.

Dr. Sally A. Santen is an ABEM oral board examiner and serves on a NBME committee. Dr. Anne Messman is on the NBME Council.

The authors declare no conflicts of interest.