Performing operations on graph through multimodal interface: An agent based architecture

Abstract

In this paper we discuss about agent based multimodal interface architecture. To illustrate this concept, we present an application to mine information from a graph data structure. This application can be used as a user friendly interface for mining information where graph can be used as data structure for information storage. The application recognizes the gestures, text based input, speech and combination of all of these. Here the various input modalities are implemented as software agents which are independent of each other. There exist MONITOR agents which will synchronize various inputs from the user and act according to it. Users response will be mapped to system vocabulary space with help of a semantic analyzer and results will be passed to MONITOR for further processing.