Sim-to-real: Six-legged Robot Control with Deep Reinforcement Learning and Curriculum Learning

Abstract

Six-Iegged robots have higher stability and balance, which helps them face more complex terrain conditions, such as sand, swamp, mine and so forth. Therefore, it is necessary to study the gait planning of six-legged robot to adapt to complex terrain. In order to control six-legged robots to adapt to different terrains, we adopt the method of deep reinforcement learning (DRL) to plan the gait of six-legged robots. The main idea is training the robot through Actor-Critic network with proximal policy optimization (PPO), in which outputs are step length, step height and orientation of the robot. This is an end-to-end approach, which tries to make the robot learn by itself and finally achieve its safe arrival to the target point through complex terrains. In order to train a good model for our robots, simplified environment is adopted to accelerate the training process. We also use curriculum learning to speed up and optimize the training. Then, we verify the reliability of the method in simulation platform and finally transfer the learned model to real robot. Our experiment shows the effectiveness of deep reinforcement learning for locomotion of six-legged robots, the acceleration of the training process by means of curriculum learning, and the improvement of the training effect.