Multimodal information retrieval, extraction and generation for use in the health domain

If an organisation such as the Department of Defence is to maximise its use of information, it has to exchange and coordinate meanings between different kinds of information sources with information from a variety of domains. A multimodal meaning base is being developed that makes it possible to identify meanings that can be expressed in linguistic and cartographic modalities. The research involves the collection of multimodal resources including structured information (databases), unstructured information, the description of the semantics, and the expression in a computational resource. The outcome of the research is the development of a multimodal information management system, HINTS, that will help solve the problems of information exchange and coordination. Our domain is communicable diseases. The design of the multimodal information management tool utilised participatory design techniques involving researchers, designers, users and stake holders. Participatory design is a comprehensive and integrated methodology for designing graphical user interfaces using low-tech materials, object oriented methods, and frequent iterative usability testing. We extended this technique to include the design of the entire system. The techniques were effective across a multidisciplinary team that included researchers and clients.