Context aided sensor and human-based information fusion

Abstract

Information fusion consists of organizing a set of data into meaningful reports to answer queries, forge a consistency story, and determine situation awareness. To provide situation understanding requires context both in information estimation and data management. In this paper, we highlight the importance of context estimation, assessment, and management to support information fusion analysis. A demonstrated example for multimodal human-based text analysis and video-based sensing and tracking is shown where context provides the basis for associating the multimodal data correlated in space and time. The use of context is demonstrated as (1) semantic text call-outs from users monitoring a video following a target for classification, (2) geographical road information from a database to locate a target, and (3) sensor-based estimation for simultaneous target tracking and classification. Context assessment and management from human-based and sensor-based sources is shown for information fusion situation awareness.