Automated visual discourse synthesis: coherence, versatility, and interactivity

In this proposal, we present comprehensive and systematic approaches of building systems that can automatically generate coherent visual discourse for interactive envirornments. A visual discourse refers to a series of connected visual displays. A coherent visual discourse requires smooth transitions between displays, consistent designs within and among displays, and effective integration of various components. Our research focuses in part on establishing a general framework by abstracting various generation systems and providing a reference model in which a specific system is considered an instantiation of the framework. In other words, any automated graphics generation system must contain a knowledge base, an inference engine, a visual realizer and an interaction handler. As a consequence, not only can a general framework serve as a template from which a specific generation system can be instantiated, but the framework also can be used as a base for comparing or evaluating different systems. We concentrate on the basic issues involved in establishing these four core components. In particular, we identify various knowledge sources and determine effective knowledge representation paradigms in constructing the knowledge base. We emphasize the efficiency, usability, and flexibility issues in modeling the inference engine. We are concerned with portability and parallelization issues in building the visual realizer, and we also take into account interaction capabilities for interactive environments. To demonstrate the generality and comprehensiveness of the framework, we address its application to the design of coherent visual discourse for heterogeneous information in interactive environments. Within such discussions, heterogeneous information refers to both quantitative and qualitative, or static and dynamic information. In addition, we assume that the system aims to support a wide variety of visual techniques, ranging from individual 2D displays to interactive 3D animation sequences. We describe a system called IMPROVISE (Illustrative Metaphor Production in Reactive Object-oriented VISual Environments) that serves as a proof-of-concept prototype. IMPROVISE is built based on our framework, aiming to automatically generate coherent visual discourse for various application domains in interactive environments. IMPROVISE has been used in two testbed application domains to demonstrate its generality and flexibility. Examples from both domains will be given to illustrate IMPROVISE’s generation process and to identify the future research areas.