Speech-reading of synthetic and natural faces: Effects of contextual cueing and mode of presentation

Abstract

A natural and a synthetic face were compared with regard to speechreading performance, with a visual and an audio-visual condition, and with three levels of contextual cueing in an experiment with 90 normal-hearing subjects. Auditory presentation (speech in noise) served as a control condition. The results showed main effects for type of face, presentation mode and contextual cueing, and also an interaction between presentation mode and type of face, such that performance was superior for the natural compared with the synthetic face, and especially in the audio-visual mode. Audiovisual speech-reading was more accurate than visual and auditory presentation and contextual cueing improved performance overall. It is suggested that a synthetic face can be a valuable and useful support if proper cueing and an auditory signal are provided, but it is outperformed by a natural face with regard to complementarity to an underspecified auditory speech signal.