Linguistic analysis of letters of recommendation for pediatric otolaryngology: A pilot study

Objectives

To analyze the language of letters of recommendation for pediatric otolaryngology fellowship for possible gender and/or racial bias. To refine methodology for future investigation of gender and language within letters of recommendation for pediatric otolaryngology fellowship.

Methods

Practicing pediatric otolaryngologists were recruited as survey participants. A total of 10 letters of recommendation were screened to represent an equal number of applicants who identify as male and female and an equal number of applicants who identify as White and applicants who identify as Black, Indigenous, or a Person of Color. Names, pronouns, location, and other identifying information were removed from the letters. Survey participants were asked to read each letter and categorize the applicant's gender and race as well as that of the letter writer. The letters were also subject to analysis by WordStat linguistic software.

Results

A total of 35 pediatric otolaryngologists participated in the survey. Participants assigned gender of the applicant correctly in 43 % of letters and gender of the letter writer correctly in 53 % of letters. Race of the applicant was judged correctly in 65 % of letters. Participants indicated words such as caring, hard-working, and prepared led them to assume the applicant was female, which was corroborated by linguistic analysis. No such words were identified as predictive of race.

Conclusion

This pilot study suggests that while there are certain linguistic associations with gender among applications to pediatric otolaryngology fellowship, they may not impact biases held by the reader. This is relevant given a recent rise in the number of female applicants to this subspecialty and ongoing unfilled fellowship positions. Future studies are needed to determine if such associations exist and ultimately affect match potential and success within the field. These studies will require analysis of more letters and with more survey respondents, along with a higher in-depth linguistic analysis.

Lay summary

This pilot study aims to evaluate letters of recommendation for pediatric otolaryngology fellowship applications for gender bias with the goal of outlining future studies. There were certain words associated with a given gender, which could impact applicants’ potential to match.

Level of evidence

NA.