A Lagrangian framework for video analytics

Abstract

The extraction of motion patterns from image sequences based on the optical flow methodology is an important and timely topic among visual multi media applications. In this work we will present a novel framework that combines the optical flow methodology from image processing with methods developed for the Lagrangian analysis of time-dependent vector fields. The Lagrangian approach has been proven to be a valuable and powerful tool to capture the complex dynamic motion behavior within unsteady vector fields. To come up with a compact and applicable framework, this paper will provide concepts on how to compute trajectory-based Lagrangian measures in series of optical flow fields, a set of basic measures to capture the essence of the motion behavior within the image, and a compact hierarchical, feature-based description of the resulting motion features. The resulting framework will bee shown to be suitable for an automated image analysis as well as compact visual analysis of image sequences in its spatio-temporal context. We show its applicability for the task of motion feature description and extraction on different temporal scales, crowd motion analysis, and automated detection of abnormal events within video sequences.