Enabling private and non-intrusive smartphone calls with LipTalk

Abstract

The typical usage for phones is making calls. However, under certain scenarios, phone calls are inappropriate or intrusive when people are having a meeting, impractical when background noise level is too high or insecure under monitoring of other personals/parties. Typical solution to these problems is to send text-based messages. Yet, we argue that the most natural, efficient way for human-beings to communicate is via speech. In this work, we provide envision for our project, a non-intrusive, convenient and secure communication system. We utilize the front camera of smart phones, efficiency of cloud computing environment and state-of-the-art text-to-speech methods to create a robust visual speech recognition system to enable people to chat with lip movements. We present the current challenges, system architecture, initial findings and planned approaches to our problem.