Using deep reinforcement learning to investigate stretch feedback during swimming of the lamprey.

Abstract

Animals have to navigate complex environments and perform intricate swimming maneuvers in the real world. To conquer these challenges, animals evolved a variety of motion control strategies. While it is known that many factors contribute to motion control, we specifically focus on the role of stretch sensory feedback. We investigate how stretch feedback potentially serves as a way to coordinate locomotion, and how different stretch feedback topologies, such as networks spanning varying ranges along the spinal cord, impact the locomotion. We conduct our studies on a simulated robot model of the lamprey consisting of an articulated spine with eleven segments connected by actuated joints. The stretch feedback is modeled with neural networks trained with deep reinforcement learning. We find that the topology of the feedback influences the energy efficiency and smoothness of the swimming, along with various other metrics characterizing the locomotion, such as frequency, amplitude and stride length. By analyzing the learned feedback networks, we highlight the importances of very local, caudally-directed, as well as stretch derivative information. Our results deliver valuable insights into the potential mechanisms and benefits of stretch feedback control and inspire novel decentralized control strategies for complex robots.