Economical Quadrupedal Multi-Gait Locomotion via Gait-Heuristic Reinforcement Learning

In order to strike a balance between achieving desired velocities and minimizing energy consumption, legged animals have the ability to adopt the appropriate gait pattern and seamlessly transition to another if needed. This ability makes them more versatile and efficient when traversing natural terrains, and more suitable for long treks. In the same way, it is meaningful and important for quadruped robots to master this ability. To achieve this goal, we propose an effective gait-heuristic reinforcement learning framework in which multiple gait locomotion and smooth gait transitions automatically emerge to reach target velocities while minimizing energy consumption. We incorporate a novel trajectory generator with explicit gait information as a memory mechanism into the deep reinforcement learning framework. This allows the quadruped robot to adopt reliable and distinct gait patterns while benefiting from a warm start provided by the trajectory generator. Furthermore, we investigate the key factors contributing to the emergence of multiple gait locomotion. We tested our framework on a closed-chain quadruped robot and demonstrated that the robot can change its gait patterns, such as standing, walking, and trotting, to adopt the most energy-efficient gait at a given speed. Lastly, we deploy our learned controller to a quadruped robot and demonstrate the energy efficiency and robustness of our method.