Efficient Object Tracking in Compressed Video Streams with Graph Cuts

In this paper we present a compressed-domain object tracking algorithm for H.264/AVC compressed videos and integrate the proposed algorithm into an indoor vehicle tracking scenario at a car park. Our algorithm works by taking an initial segmentation map or bounding box of the target object in the first frame of the video sequence as input and applying Graph Cuts optimization based on a Markov Random Field model. Our algorithm does not rely on pixels (except for the first frame) and works by only using the codec motion vectors and block coding modes extracted from the H.264/AVC bitstream via inexpensive partial decoding. In this way, we manage to reduce the compute and storage requirements of our system significantly compared to “pixel-domain” tracking algorithms that first fully decode the video stream and work on reconstructed pixels. We demonstrate the quantitative performance of our algorithm over VOT2016 dataset and also integrate our algorithm into a camera-based parking management system and show qualitative results in a real application scenario. Results show that our compressed-domain algorithm provides a good compromise between high accuracy tracking and low-complexity processing showing that it is feasible for scenarios requiring large-scale object tracking in bandwidth-limited conditions.