Examining the attribution of gender and the perception of emotions in virtual humans

Abstract

Virtual Humans (VHs) are becoming increasingly realistic, raising questions about how users perceive their gender and emotions. In this study, we investigate how textually assigned gender and visual facial features influence both gender attribution and emotion recognition in VHs. Two experiments were conducted. In the first, participants evaluated a nonbinary VH animated with expressions performed by both male and female actors. In the second part, participants assessed binary male and female VHs animated by either real actors or data-driven facial styles. Results show that users often rely on textual gender cues and facial features to assign gender to VHs. Emotion recognition was more accurate when expressions were performed by actresses or derived from facial styles, particularly in nonbinary models. Notably, participants more consistently attributed gender according to textual cues when the VH was visually androgynous, suggesting that the absence of strong gendered facial markers increases the reliance on textual information. These findings offer insights for designing more inclusive and perceptually coherent virtual agents.