Combining Reinforcement Learning and Lazy Learning for Faster Few-Shot Transfer Learning

Abstract

Since repeating a task with a humanoid robot many times is typically far too time consuming and strenuous for the robotic mechanism, learning is often shifted to simulation. Bridging the sim-to-real gap, however, still requires considerable real-world effort. In this paper we explore how to reduce the number of required repetitions with a novel few-shot transfer learning methodology. The skill is initially encoded with a deep neural network in one domain, and later adapted for a different target domain by re-training only a partllayer of this deep neural network with real data. For retraining we propose to combine lazy learning and reinforcement learning. Our experiments show that such combination is considerably faster than only using either one of these and an order of magnitude faster than learning from scratch. We demonstrated the approach on the example of robotic throwing, a complex dynamic skill where the outcome of the task is not explicitly dependent on the final position of the robot motion. The experiments were performed for sim-to-sim transfer learning on the full-sized humanoid robot TALOS, with initial throwing implementation on the real robot.