Automatic pan-tilt-zoom calibration in the presence of hybrid sensor networks

Wide-area context awareness is a crucial enabling technology for next generation smart buildings and surveillance systems. It is not practical to cover an entire building with cameras, however it is difficult to infer missing information when there are significant gaps in coverage. As a solution, we advocate a class of hybrid perceptual systems that builds a comprehensive model of activity in a large space, such as a building, by merging contextual information from a dense network of ultra-lightweight sensor nodes with video from a sparse network of high-capability sensors. In this paper we explore the task of automatically recovering the relative geometry between a pan-tilt-zoom camera and a network of one-bit motion detectors. We present results for the recovery of geometry alone, and also recovery of geometry jointly with simple activity models. Because we don't believe a metric calibration is necessary, or even entirely useful for this task, we formulate and pursue the novel goal we term functional calibration. Functional calibration is the blending of geometry estimation and simple behavioral model discovery. Accordingly, results are evaluated in terms of the ability of the system to automatically foveate targets in a large, non-convex space, not in terms of pixel reconstruction error.