Human-machine performance configuration for multidimensional and multi-modal interaction in virtual environments

Abstract

The paper presents a human machine performance configuration for multidimensional and multi modal interaction in virtual environments. A performer is described as an observer interacting with a virtual environment to extract information in a time critical manner. In the presented research, a performer's multi modal capacity is supported by time scheduling techniques for parallel processing of sensors and displays to provide synchronous perceptual feedback. This modality is coupled to multidimensional numerical simulations. A software architecture facilitates a temporal framework for interactivity, complementary to static spatial organization of geometry for graphical display. Mutually applicable design criteria include the management of computing resources, a configuration of an observation space, and VR authoring. Accordingly the following system designs are introduced: 1) criteria for bounded synchronization among parallel processes; 2) an interface paradigm implemented with the criteria to facilitate spatio temporal articulation of multidimensional control signals extracted from continuous real time gestures of observers; 3) a run time configuration protocol implemented to support an efficient VR authoring capacity. Parallel processes are represented as nodes in a directed graph, with intelligent edges that determines how services between nodes are to be managed.