A Multimodal Audio Visible and Infrared Surveillance System (MAVISS)

Abstract

This paper presents a low cost surveillance system employing multimodal information (visible, infrared and audio signals) for monitoring small area and detecting alarming events. To ensure efficient and robust operation, the system captures different aspects of the environment using audio and video information. Infrared imagery is usedfor night and other low level lighting situations. The visual processing module of the system uses a motion based approach for detecting objects, and employs Kalman filter model for tracking its motion. Environmental sound is recognized by processing audio signals to extract features in the form of Mel-Frequency Cepstral coefficients (MFCC), which are then used for classification by Dynamic Time Warping (DTW) technique. Semantic rules are proposed to identify alarming events by using information from audio and video module. Experimental results are shown on some typical sequences and publicly available dataset.