Integrated model of cerebellal supervised learning and basal ganglia’s reinforcement learning for mobile robot behavioral decision-making

Abstract

Behavioral decision-making in unknown environments of mobile robots is a crucial research topic in robotics. Inspired by the working mechanism of different brain regions in mammals, this paper designed a new hybrid model integrating the functions of cerebellum and basal ganglia by simulating the memory replay of hippocampus, so as to realize the autonomous behavioral decision-making of robot in unknown environments. A reinforcement learning module based on Actor-Critic framework and a developmental network module are used to simulate the functions of the basal ganglia and cerebellum, respectively. Considering the different functions of D1 and D2 dopamine receptors in basal ganglia, an Actor network module with separate learning of positive and negative rewards is designed for the basal ganglia to realize efficient exploration of the environments by the agent. According to the characteristics of biological memory, a physiological memory priority index is designed for hippocampus memory replay, which improves the offline learning efficiency of cerebellum. The integrated model enables dynamic switching between decisions made by cerebellum and basal ganglia based on the agent’s cognitive level with respect to the environment. Finally, the effectiveness of the proposed model is verified through experiments on agent navigation in both simulation and real environments, as well as through performance comparison experiments with other learning algorithms.