Investigating accuracy disparities for gender classification using convolutional neural networks

Abstract

Automatic gender recognition (AGR) is a subfield of facial recognition that has recently been scrutinized for bias in the form of misgendering and erasure against various identity groups in our society. Recent studies have found that several commercial AGR classifiers (from Microsoft, IMB, Face++) are biased against women and darker-skinned people as well as gender non-binary people [8, 11]. In this work, we investigate and quantify AGR classifier bias against transgender people by developing and evaluating three different convolutional neural networks (CNN): using images of cisgender individuals, using images of transgender individuals, and using images of both cisgender and transgender individuals. We find that the cisgender trained classifier is 91.7% accurate when evaluated on cisgender people, but only 68.9% accurate when evaluated on transgender people, with the worst performance of 38.6% precision for transgender men. We investigate this low precision further by performing additional experiments where various parts of the face are obscured. We end with recommendations for commercial classifiers based upon our findings.