IMKA: a multimedia organization system combining perceptual and semantic knowledge

Abstract

In the demo, we present the IMKA system, which implements the innovative approach of integrating perceptual information such as low-level features and images, and symbolic information such as words to represent the knowlege associated with a large multimedia collection for multimedia organization and retrieval. The IMKA system utilizes the unique MediaNet framework, which greatly extends existing knowlege representation tools in the text domain (e.g., semantic networks and WordNet) and the multimedia domain (e.g. Multimedia Thesaurus) by combining perceptual and semantic concepts in the same network and by supporting perceptual and semantic relationships among concepts exemplified by different media. It also brings the level of multimedia retrieval closer to users' needs by translating low-level feature queries to high-level semantic queries and vice versa. We will demonstrate the process of constructing the MediaNet knowledge base and new ways of searching multimedia in the IMKA system by presenting the current implementation of the IMKA system that uses image collections from online sources.