Learn to climb: teaching a reinforcement learning agent the single rope ascending technique

Single rope ascending technique is used in industrial alpinism, forestry, or various leisure activities. This paper presents a multi-body model of this technique involving an actuated 3D model of a humanoid, the climbing gear, and the rope, modeled as a finite-element object. This model serves as a training ground for reinforcement learning agents trying to mimic humans in rope climbing. To demonstrate the environment, an agent with a state-of-the-art reinforcement learning algorithm (Soft Actor-Critic) was trained. Results suggest that the agent can learn how to ascend the rope with speed comparable to real humans. However, the learned technique is not human-like: the artificial agent uses its arms excessively to climb, which would be too tiring for a human. That is because the environment only rewards ascension and does not penalize the energy used. The presented learning environment is developed for humanoid robots in mind that can perform complex tasks while on the rope and can carry much heavier payloads compared to climbing robots in the literature.