The ALFRESCO interactive system (abstract)

Abstract

This work is aimed at building a dialogue system in which natural language is the basic communication channel, but the computer is seen as an active agent that allows a multimedia type interaction. In this way the means of communication are amplified, with the possibility of refering to images and other texts. ALFRESCO is an interactive system for a user interested in frescoes. It runs on a SUN 4 connected to a videodisc unit and a touchscreen. The particular videodisc in use includes images about Fourteenth Century Italian frescoes and monuments. The system, beside understanding and using language, shows images and combines film sequences. Images are active in that the user may refer to items by combining pointing with the use of linguistic demonstratives; for example, the user can point to a detail of a fresco and say “can I see another painting representing thisA saint ?“ Also, the system’s linguistic output includes buttons that allow the user to enter in an hypertextual modality. The dialog may cause zooming into details or changing the focus of attention into other frescoes. The overall aim is not only to provide information, but also to promote other masterpieces that may attract the user. The knowledge of the system is represented as: ● a KB expressed in a KLOne-like language used for defining everything the system can reason about frescoes, monuments, painters, contents of frescoes, towns etc. and providing the base for ALFRESCO’s deductive inference capabilities; c a NoteCards hypermedia network containing unformalized knowledge such as art critics’ opinions on the paintings and their authors. Permission to copy without fee all or part of this material is grantad provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. Q 1993 ACM 0-89791 -575-5193 /000410523 . ..S1 .50 The system is based on several linguistic modules such as: a) a chart-based parser able to deal with flexible expressions and in particular idiomatic forms and some kinds of ill-formed inpu~ b) a semantic analyzer able to disambiguate the sentence in the given domain through interaction with the parseq c) a component that builds the logical form interacting with the KB; d) a topic component that takes into account also deixis (references to images); e) a pragmatic component, substantially based on a model of the interest of the use~ f.) a natural language generator that takes into account the user’s interest model. The generated output is in the form of an hypertextual card the text is enriched with dynamically generated buttons that the user can click to get more information and explore “the surroundings”. The interaction of hypermediality with AI and NLP technology opens a wide range of new perspectives for the development of intelligent interfaces, especially in the art and cultural domain we believe that it will ultimately lead to an innovative paradigm for man-machine interaction.