Learn from Interaction: Learning to Pick via Reinforcement Learning in Challenging Clutter

Bin picking is a challenging problem in robotics due to high dimensional action space, partially visible objects, and contact-rich environments. State-of-the-art methods for bin picking are often simplified as planar manipulation, or learn policy based on human demonstration and motion primitives. The designs have escalated in complexity while still failing to reach the generality and robustness of human picking ability. Here, we present an end-to-end reinforcement learning (RL) framework to produce an adaptable and robust policy for picking objects in diverse real-world environments, including but not limited to tilted bins and corner objects. We present a novel solution to incorporate object interaction in policy learning. The object interaction is represented by the poses of objects. The policy learning is based on two neural networks with asymmetric state inputs. One acts on the object interaction information, while the other acts on the depth observation and proprioceptive signals of robots. The results of experiment shows remarkable zero-shot generalization from simulation to the real world and extensive real-world experiments show the effectiveness of the approach.