To Blind or Not to Blind: Evaluating the Impact of Withholding Scores and Grades From Interviewers in General Surgery Resident Recruitment.

Abstract

Objective: Compare scoring outcomes between interviewers blinded to scores/grades/MSPE and those with the full applicant file to evaluate the effect of blinding on interview scores and ensure applicants can be confidently evaluated when blinding is used.

Design, setting and participants: Nineteen interviewers were purposively randomized to receive a complete application or file with all information except applicant grades/MSPE/USMLE score(s) of 90 applicants prior to 218 interviews during 2022 to 2023 general surgery recruitment. Blinding was randomly assigned while ensuring blinded and nonblinded interviews for both interviewers and applicants. Two program leaders involved in study implementation were excluded from blinding. All other aspects of the selection process remained unchanged from historic methods. Each applicant had 3 to 4 interviews. Each interview was scored prior to discussion with other faculty using a 10-point scale. Descriptive and univariate statistics analyzed scoring patterns. Qualitative data regarding the experiences of blinded interviewers was analyzed to generate themes.

Results: There were no differences in interview scores or difference from the applicants' mean scores between blinding groups. This remained true for within-applicant analyses and for all but 1 interviewer (95%) for within-interviewer analyses. Between-interviewer score differences were seen for interview scores across all interviewers and when comparing nonblinded vs. nonblinded scores across interviewers, but not when comparing blinded vs. blinded scores across interviewers. Qualitative data support the ability to confidently evaluate interview performance when blinded, frequent practice of "self-blinding" to limit bias even when given scores/grades/MSPE, and belief that scores/grades/MSPE are relevant for screening, but the interview has separate priorities.

Conclusions: Blinding of interviewers to scores/grades/MSPE did not significantly change interview scoring outcomes. Interviewer experiences support the ability to confidently evaluate interview performance when blinded. Given that negative effects of blinding were not found and prior work supports that bias may be mitigated by blinded interviews, we support its use in residency recruitment.