HumanoidMamba: Generalized Mamba-Based Policy Learning With Next-Action Prediction for Humanoid Locomotion

Abstract

In various human-centered unstructured environments, humanoid robots tend to exhibit superior adaptability and operational performance relative to other types of robots. Existing approaches to whole-body control in humanoid robots can be primarily composed of model-based and learning-based methods. While model-based methods rely heavily on manual design, current state-of-the-art (SOTA) learning-based methods employ Transformers as backbone networks, often leading to suboptimal computational efficiency. Inspired by the demonstrated superiority of Mamba over Transformers in large language models (LLMs) and computer vision domains, HumanoidMamba is proposed as a framework that leverages Mamba as the core backbone of the policy network to generate humanoid locomotion. Mamba, serving as the backbone, improves computational efficiency compared to Transformers while maintaining the capacity of the model for content-based reasoning. In addition, analogous to next-token prediction in LLMs, a modality-aligned prediction approach is introduced for generating the next action in the policy network, referred to as next-action prediction. The modality-aligned next-action prediction approach establishes a paradigm that not only enhances the understanding of temporal information by the policy but also facilitates extension to handle trajectories with missing modalities. Experimental results substantiate that the proposed HumanoidMamba consistently generates humanoid locomotion characterized by exceptional stability and robustness. Beyond surpassing existing SOTA methods across multiple performance metrics, HumanoidMamba further unveils remarkable generalization capability by adapting to velocity commands that are absent from the expert demonstration trajectories.