Hierarchical Reinforcement Learning for Quadrupedal Robots: Efficient Object Manipulation in Constrained Environments.

Abstract

This study introduces a hierarchical reinforcement learning (RL) framework tailored to object manipulation tasks by quadrupedal robots, emphasizing their real-world deployment. The proposed approach adopts a sensor-driven control structure capable of addressing challenges in dense and cluttered environments filled with walls and obstacles. A novel reward function is central to the method, incorporating sensor-based obstacle observations to optimize the decision-making. This design minimizes the computational demands while maintaining adaptability and robust functionality. Simulated trials conducted in NVIDIA Isaac Sim, utilizing ANYbotics quadrupedal robots, demonstrated a high manipulation accuracy, with a mean positioning error of 11 cm across object-target distances of up to 10 m. Furthermore, the RL framework effectively integrates path planning in complex environments, achieving energy-efficient and stable operations. These findings establish the framework as a promising approach for advanced robotics requiring versatility, efficiency, and practical deployability.