Use of visual and tactile data for generation of 3-D object hypotheses

Most existing 3-D object recognition/localization systems rely on a single type of sensory data, although several sensors may be available in a robot task to provide information about the objects to be recognized. In this paper, the authors present a technique to localize polyhedral objects by integrating visual and tactile data. It is assumed that visual data is provided by a monocular visual sensor, while tactile data by a planar-array tactile sensor in contact with the object to be localized. The authors focus on using tactile data in the hypothesis generation phase to reduce the requirements of visual features for localization to a V-junction only. The main concept of this technique is to compute a set of partial pose hypotheses off-line by utilizing tactile data, and then complement these partial hypotheses on-line using visual data. The technique presented is tested using simulated and real data.<>