The Dunning‒Kruger effect in resident predicted and actual performance on the American Board of Emergency Medicine in-training examination

Objectives

The Dunning–Kruger effect (DKE) is a cognitive bias wherein individuals who are unskilled overestimate their abilities, while those who are skilled tend to underestimate their capabilities. The purpose of this investigation is to determine if the DKE exists among American Board of Emergency Medicine (ABEM) in-training examination (ITE) participants.

Methods

This is a prospective, cross-sectional survey of residents in Accreditation Council for Graduate Medical Education (ACGME)-accredited emergency medicine (EM) residency programs. All residents who took the 2022 ABEM ITE were eligible for inclusion. Residents from international programs, residents in combined training programs, and those who did not complete the voluntary post-ITE survey were excluded. Half of the residents taking the ITE were asked to predict their self-assessment of performance (percent correct), and the other half were asked to predict their performance relative to peers at the same level of training (quintile estimate). Pearson's correlation (r) was used for parametric interval data comparisons and a Spearman's coefficient (ρ) was determined for quintile-to-quintile comparisons.

Results

A total of 7568 of 8918 (84.9%) residents completed their assigned survey question. A total of 3694 residents completed self-assessment (mean predicted percentage correct 67.4% and actual 74.6%), with a strong positive correlation (Pearson's r 0.58, p < 0.001). There was also a strong positive correlation (Spearman's ρ 0.53, p < 0.001) for the 3874 residents who predicted their performance compared to peers. Of these, 8.5% of residents in the first (lowest) quintile and 15.7% of residents in the fifth (highest) quintile correctly predicted their performance compared to peers.

Conclusions

EM residents demonstrated accurate self-assessment of their performance on the ABEM ITE; however, the DKE was present when comparing their self-assessments to their peers. Lower-performing residents tended to overestimate their performance, with the most significant DKE observed among the lowest-performing residents. The highest-performing residents tended to underestimate their relative performance.