An algorithm for 3D scene description in an unknown environment

Scene description plays a major role in the interpretation of images. In the paper a novel data- and rule-driven system for 3D segmentation and scene description in an unknown environment is presented. This system generates hierarchies of features that correspond to structural elements such as boundaries and shape classes of individual objects as well as relationships between objects. It is implemented as an added high-level component to an existing low-level binocular vision system (Don and He, 1988). Based on a pair of matched stereo images produced by that system, 3D segmentation is first performed to group object boundary data into several edge-sets each of which is believed to belong to one particular object. Then gross features of each object are extracted and stored in an object record. The final structural description of the scene is accomplished with information in the object record, a set of rules and a rule implementor. The system is designed to handle partially occluded objects of different shape and size on the 2D images. Experimental results have shown its success in computing both object and structural level descriptions of common man-made objects.<>