Perception-based search and manipulation in a semi-structured environment

In this paper we present two new approaches developed in the context of autonomous search for localization and manipulation of books in a semi-structured environment, like a library. Search for books is done by means of visual information which is also integrated with force sensing for book manipulation. Interest regions are located using a local segmentation algorithm with automatic threshold selection. From the extracted visual features, we have implemented a fast tracking method by means of model-based matching. We also apply some recognition techniques in order to find the desired book and to guide the gripper over it. Grasping is finally done by integrating visual and force perception into a global hybrid control law. We have applied the implemented algorithms to the UJI librarian robot and results show the robustness of the implemented search algorithm as well as its fast performance. With respect to the grasping system, vision and force coupling are shown to be an adequate combination of sensors for book manipulation