Multiple appearance models for face tracking in surveillance videos

Face tracking is a key component for automated video surveillance systems. It supports and enhances tasks such as face recognition and video indexing. Face tracking in surveillance scenarios is a challenging problem due to ambient illumination variations, face pose changes, occlusions, and background clutter. We present an algorithm for tracking faces in surveillance video based on a particle filter mechanism using multiple appearance models for robust representation of the face. We propose color based appearance model complemented by an edge based appearance model using the Difference of Gaussian (DOG) filters. We demonstrate that combined appearance models are more robust in handling the face and scene variations than a single appearance model. For example, color template appearance model is better in handling pose variations but they deteriorate against illumination variations. Similarly, an edge based model is robust in handling illumination variations but they fail in handling substantial pose changes. Hence, a combined model is more robust in handling pose and illumination changes than either one of them by itself. We show how the algorithm performs on a real surveillance scenario where the face undergoes various pose and illumination changes. The algorithm runs in real-time at 20 fps on a standard 3.0 GHz desktop PC.