Scene augmentation via the fusion of industrial drawings and uncalibrated images with a view to marker-less calibration

Abstract

The application presented augments uncalibrated images of factories with industrial drawings. Industrial drawings are among the most important documents used during the lifetime of industrial environments. They are the only common documents used during design, installation, monitoring and control, maintenance, update and finally dismantling of industrial units. Leading traditional industries towards the full use of virtual and augmented reality technology is impossible unless industrial drawings are integrated into our systems. We provide the missing link between industrial drawings and digital images of industrial sites. On one hand, this could enable us to calibrate cameras and build a 3D model of the scene without using any calibration markers. On the other hand it brings industrial drawings, floor map, images and 3D models into one unified framework. This provides a solid foundation for building efficient enhanced virtual industrial environments. The augmented scene is obtained by perspective warping of an industrial drawing of the factory onto its floor, wherever the floor is visible. The visibility of the floor is determined using probabilistic reasoning over a set of clues including (1) floor color/intensity (2) image warping and differencing between an uncalibrated stereoscopic image pair using the ground plane homography. Experimental results illustrate the approach.