Multi-view calibration from planar motion for video surveillance

We present a technique for the registration of multiple surveillance cameras through the automatic alignment of image trajectories. The algorithm address the problem of recovering the relative pose of several stationary cameras that observe one or more objects in motion. Each camera tracks several objects to produce a set of trajectories in the image. Using a simple calibration procedure, we recover the relative orientation of each camera to the local ground plane in order to projectively unwarp image trajectories onto a nominal plane of correct orientation. Unwarped trajectory curves are then matched by solving for the 3D to 3D rotation, translation, and scale that bring them into alignment. The relative transform between a pair of cameras is derived from the independent camera-to-ground-plane rotations and the plane-to-plane transform computed from matched trajectories. Registration aligns n-cameras with respect to each other in a single camera frame (that of the reference camera). The approach recovers both the epipolar geometry between all cameras and the camera-to-ground rotation for each camera. After calibration, points that are known to lay on a world ground plane can be directly backprojected into each of the camera frames. The algorithm is demonstrated for two-camera and three-camera scenarios by tracking pedestrians as they move through a surveillance area and matching the resulting trajectories.