Control of FES using reinforcement learning: accelerating the learning rate

Abstract

Prior knowledge can be used to accelerate the process of reinforcement learning. An adaptive fuzzy logic controller designed to control the swing phase of paraplegic gait was trained on a computer model using reinforcement learning. Instead of starting from scratch with generic fuzzy rules, the controller was jump-started in two different ways with experienced rules. First, supervised learning was used to initially train the controller, then two system parameters were altered and the reinforcement learning algorithm proceeded to find an optimal solution. This required a total of 34 simulation cycles. The same task, using reinforcement learning alone, required almost 150 cycles. Second, the trained controller was transferred to two individuals of differing body mass and height. It required less than 20 additional cycles to converge in both cases. By placing the controller initially closer to an optimal solution, jump-starting greatly reduces the number of simulation cycles required.