Discriminative Fields for Modeling Semantic Concepts in Video

A current trend in video analysis research hypothesizes that a very large number of semantic concepts could provide a novel way to characterize, retrieve and understand video. These semantic concepts do not appear in isolatation to each other and thus it could be very useful to exploit the relationships between multiple semantic concepts to enhance the concept detection performance in video. In this paper we present a discriminative learning framework called Multi-concept Discriminative Random Field (MDRF) for building probabilistic models of video semantic concept detectors by incorporating related concepts as well as the low-level observations. The proposed model exploits the power of discriminative graphical models to simultaneously capture the associations of concept with observed data and the interactions between related concepts. Compared with previous methods, this model not only captures the co-occurrence between concepts but also incorporates the raw data observations into a unified framework. We also present an approximate parameter estimation algorithm and apply it to the TRECVID 2005 data. Our experiments show promising results compared to the single concept learning approach for semantic concept detection in video.