Experience retrieval in a ubiquitous home

Abstract

We present a system for retrieval and summarization of continuously archived multimedia data from a home-like ubiquitous environment. Data from pressure-based floor sensors are analyzed to index video and audio from a large number of sources. Video and audio handover are implemented to retrieve continuous video streams with sound as a person is moving in the environment. Key frame extraction is proposed and several algorithms are implemented to obtain compact summaries corresponding to the activity of each person. Clustering algorithms and image analysis are used to identify actions and events. The system consists of a graphical user interface that can be used to retrieve video summaries interactively using simple queries.