Reinforcement Learning for Exploration vs. Exploitation Problems in Medical Exercise Sessions

Abstract

Biofeedback in gamified medical exercise sessions has proven to be an effective technique for adapting patient behaviours to improve health outcomes. In this paper, we formulate a method for designing optimal training sessions with two conflicting goals: maximising the desired exercise effect and sufficiently exciting the system for identification in order to update the personalised patient model. We exploit the flexibility of model-free reinforcement learning to obtain an optimal controller, which is robust to uncertainty in the system parameters. We compare the controller from reinforcement learning to a standard dual control formulation in simulation on an illustrative case study of building pelvic floor muscular strength and tone while performing Kegel exercises. The results indicate that the reinforcement learning method attains a better exercise effect while improving the parameter estimates compared to a standard dual controller. However, due to the trial-and-error nature of reinforcement learning, this comes at the expense of computational time.