Lessons from speechreading

Speechreading is the ability to understand a speaker’s thoughts by watching the movements of the face and body and by using the information provided by the situation and the language. People with normal hearing and the hearing impaired use speechreading to augment communication especially in noisy environments. Just as people learn this skill, machines can be trained to understand a speakers meaning. Audio-Visual Automatic Speech Recognition (AV ASR) systems use audio and visual information to recognize what has been ‘said’. The speech sounds and movements provided need not be standard speech sounds or movements. The system will provide recognition given audio information only, visual information only or both.