An Implementation of Multimodal User Interface using Speech, Image and EOG

There have been many recent studies on gaze recognition system in the field of HCI (Human Computer Interaction). This system will be the most natural and intuitive HCI system due to the application of gaze direction or biomedical Signals. We propose a multimodal user interface system using the nine directional gaze recognition based on image, EOG (Electrooculography) signal and speech recognition. In this paper, we use DFA (Deterministic Finite Accepter), Haar-like feature and Adaboost algorithm, SVM (Support Vector Machine) for gaze recognition based on the EOG signal and image. Furthermore, the CHMM (Continuous Hidden Markov Model) based speech recognition system has been linked for inputting speech commands. The proposed multimodal user interface system solves the problem of constraint of single modal recognition systems. As a result, the proposed system achieves higher recognition performance and more natural interface using speech commands.