Visual Servoing Using Virtual Space for Both Learning and Task Execution

In this paper, we describe a framework for per-forming an object picking task using visual servoing. As a robotic manipulator approaches the object to be grasped, a convolutional neural network (CNN) is used to generate motions to utilize visual servoing. However, to obtain an appropriate CNN, it is necessary to prepare a large amount of training data. Therefore, we propose a method that utilizes a virtual environment to reduce the load. Moreover, while performing the actual object picking task, sensor data acquisition and motion generation are performed using the virtual environment. This renders approaching the object possible even when the texture changes in the actual environment where the robot moves. An object grasping experiment was conducted on a rectangular box or a cylindrical object, and the performance of the proposed framework was verified.