Behavior recognition from video based on human constrained descriptor and adaptable neural networks

Abstract

In this paper we introduce a new descriptor, the Human Constrained Pixel Change History (HC-PCH), which is based on Pixel Change History (PCH) but focuses on the human body movements over time. We propose a modification of the conventional PCH which entails the calculation of two probabilistic maps, based on human face and body detection respectively. The features extracted from this descriptor are used as input to an HMM-based behavior recognition framework. We also introduce a rectification framework of behavior recognition and classification by incorporating an expert user's feedback into the learning process through two proposed schemes: a plain non-linear one and an adaptable one, which requires fewer training samples and is more effective in decreasing misclassification error. The methods presented are validated on a real-world computer vision dataset comprising challenging video sequences from an industrial environment.