Multi-sensor registration for objects motion detection

Abstract

The first step in order to achieve low-level multi-sensor fusion is the registration of images from multiple types of sensors. This is a very important task: it can be useful to improve the detection or the tracking of a moving object in an area. Putting together the information of an IR (infrared) and a visual camera we can use the information of the heat emanated from a human body to detect a pedestrian in the video. Basically we can align the IR and visual data knowing the calibration of the sensors, and always moving them together. In a real situation, it can be useful to align the images without imposing anything on the starting condition of the cameras and their relative position. In this paper, we present a method to automatically register IR with visual image data. The method uses geometric structures that are matched with a partial graph matching algorithm. We also introduce an iterative method to map IR and visual sequences using the homography matrix between frames. This method can be used to improve the multi-sensor motion detection: from an initial detection of a moving object in the visual image we can obtain the corresponding region in the thermal image.