Encoding flexible gait strategies in stick insects through data-driven inverse reinforcement learning.

Abstract

Stick insects exhibit remarkable adaptive walking capabilities across diverse environments; however, the mechanisms underlying their gait transitions remain poorly understood. Although reinforcement learning (RL) has been employed to generate insect-like gaits, the design of an appropriate reward function presents a challenge due to the probabilistic and continuous nature of gait transitions. This study utilized maximum entropy inverse RL to infer the reward function that governs stick insect gait selection, incorporating walking dynamic parameters-namely, velocity, direction, and acceleration-alongside antenna joint movements as state variables. By analyzing the inferred reward structures, we clarified the underlying principles that drive gait transitions and emphasized the role of sensory feedback in gait modulation. The efficacy of the inferred policies was validated through an assessment of their ability to reproduce expert trajectories, demonstrating that stick insect gaits can be learned from observable states during locomotion. Furthermore, interspecies variations and noncanonical gait patterns were examined, providing insights into the flexibility and adaptability of insect locomotion. This data-driven approach offers a biologically interpretable framework for gait modeling and contributes to bioinspired robotic design by facilitating adaptive control strategies for hexapod robots.