Simultaneously Determining Target Object and Transport Velocity for Manipulator and Moving Vehicle in Piece-Picking Operation

We propose a deep learning-based method that simultaneously determines a target object to be picked up by an autonomous manipulator and the velocity of an automated guided vehicle (AGV) that passes in front of the manipulator while the AGV carries a carton case containing the target and other objects. Our method can efficiently perform automated piece-picking operations in warehouses without the AGV needing to pause in front of the manipulator. In our method, for preparing supervised data sets with color images of objects that are randomly piled up in the carton case, a simulator checks whether each object is “pickable” or not by trying to plan the manipulator’s motion to have its hand reach the object while avoiding surrounding obstacles by using the depth images in consideration of the carton case’s movement and velocity. Then, we make each of multiple deep convolutional neural networks (DCNNs) corresponding to multiple levels of velocity learn to detect grasp points for only pickable objects from an RGB image. In our experimental test, using our method, a prototype of the system successfully picked ordered objects up without the AGV pausing while the AGV changed its velocity depending on the layout of the objects in the carton case.