Understanding the role of standardized letters of recommendation in reducing gender bias for applicants to the neurosurgery match 2020.

Abstract

Objective: Women are underrepresented in neurosurgical training, with progressively fewer women in successive ranks. Letters of recommendation (LORs) are crucial for granting residency interviews, but data suggest they are biased by the applicant's gender, with women applicants receiving lower-quality letters. This study examines whether the standardized LOR (SLOR) format, introduced in the 2020 neurosurgery match, reduces gender bias.

Methods: The authors reviewed Electronic Residency Application Service applications submitted to a single neurosurgery training program in 2020. LORs were separated by applicant gender and subjected to quantitative linguistic analysis using a custom MATLAB script. LORs were assessed for length, classification as letters of minimal assurance, and gendered stereotypic adjectives. An LOR provides an overall assessment of a candidate's potential and typically includes a description of the writer's relationship to the applicant, a description of intent as demonstrated by the record of the applicant, and a review of the applicant's accomplishments. Omission of any of these three components gives the reader only a partial grasp of a candidate's qualifications, resulting in a letter of minimal assurance. Gendered stereotypic adjectives are the result of a significant body of research devoted to analysis of gender differences in language. These adjectives fall within five word groups: standout, ability, research, grindstone, and teaching, with the latter two categories more often being used to describe females. SLORs were analyzed by noting the percentile ranking in each category. United States Medical Licensing Examination Step 1 scores were compared. Univariate analysis was performed for each LOR variable, using two-tailed t-tests. The distribution of SLOR percentile rankings was evaluated using a Kolmogorov-Smirnov test. A p value < 0.05 was considered significant. Lastly, match outcome data were analyzed to assess impact on outcome.

Results: The authors reviewed 910 narrative LORs (NLORs) and 566 SLORs. Of the applicants, 71% were male and 29% were female. There were no significant differences in LOR length, total letters per applicant, or Step 1 scores by gender. Female applicants were more likely to have received letters of minimal assurance (5% vs 1%) and be described by teaching words (p = 0.0038). This difference persisted when evaluating narrative portions from SLORs alone. There were no differences in percentile rankings or match rates.

Conclusions: Women receive more letters of minimal assurance and are more likely to be described with teaching words in NLORs, indicating a gender bias. Whether this influences match outcome is difficult to determine based on poor match data, but the present research suggests it does not. Nevertheless, SLORs may help to reduce the identified narrative bias.