Feature extraction using gaze of participants for classifying gender of pedestrians in images

Abstract

Human participants look at informative regions when attempting to identify the gender of a pedestrian in images. In our preliminary experiment, participants mainly looked at the head and chest regions when classifying gender in these images. Thus, we hypothesized that the regions in which participants gaze locations were clustered would contain discriminative features for a gender classifier. In this paper, we discuss how to reveal and use gaze locations for the gender classification of pedestrian images. Our method acquired the distribution of gaze locations from various participants while they manually classified gender. We termed this distribution a gaze map. To extract discriminative features, we assigned large weights to regions with clusters of gaze locations in the gaze map. Our experiments show that this gaze-based feature extraction method significantly improved the performance of gender classification when combined with either a deep learning or a metric learning classifier.