Imitation-constrained evolutionary learning for multi-locomotion skill of bipedal wheeled robots in unstructured terrains

Bipedal wheeled robots present significant locomotion challenges in unstructured terrains due to inherent dynamic instability and under-actuation characteristics. While reinforcement learning offers promising solutions for mobility enhancement, achieving multi-skill generalization and avoiding local optima in policy optimization remains problematic. This paper proposes an imitation-constrained evolutionary reinforcement learning framework that reformulates multi-locomotion skill acquisition as an evolutionary process. The methodology is characterized by: (1) Task-specific sub-policies with simplified reward functions ensuring local optimality, (2) Expert-student architecture where optimized sub-policies serve as expert models, and (3) Dynamic imitation-constrained Reinforcement Learning strategy enabling progressive skill inheritance during new task learning. The evolutionary mechanism guarantees global optimality of multi-locomotion skill learning through a single agent. Experimental validation demonstrates 192 % cumulative reward improvement compared to end-to-end learning paradigms. Through dynamic parameter randomization, the control strategy was successfully implemented on physical prototypes, achieving zero-shot sim-to-real transfer with multimodal adaptation across muddy rock, grassland, and discrete obstacle terrains.