Seeing with your hands: A better way to obtain perception capabilities with a personal robot

Abstract

For validation and to visualize the advantage of using an in hand Kinect sensor we are using the PR2 robot to observe a cluttered tabletop environment, as shown in Fig. 4. We assume the robot has to remain stationary, since positions around the table are not accessible to the robot. To show the advantage of an in hand camera we conduct two observation attempts, one with only the head mounted Kinect sensor and the other with the Kinect sensor held in the robots right gripper. The motion of the head mounted Kinect sensor is restricted by a pan/tilt joint, while the in hand sensor can be positioned in the entire manipulator workspace. The result of the observed environment, represented as an occupancy grid, using only the head mounted Kinect can be seen in Fig. 5a. The head mounted Kinect was able to observe 87% of the scene, however substantial parts of the space are still unobserved. In comparison, by using the in hand sensor the robot is able to observe 97% of the environment. We can see that the in hand camera increases observation ability of the robot, which in turn decreases the unobserved space by 10%. Objects previously unobserved and hidden are visible due to the increase in possible sensor positions which is especially critical for manipulation tasks, collision avoidance, object search and so on.