Reinforcement Learning Identifies Age-Related Balance Strategy Shifts

Abstract

Falls are one of the leading causes of non-disease death and injury in the elderly, partly due to the loss of muscle mass in a musculoskeletal disorder named sarcopenia. Studying the impact of this muscle weakness on standing balance through direct human experimentation poses ethical dilemmas, involves high costs, and fails to fully capture the internal dynamics of the muscle. To address these limitations, we employ neuromusculoskeletal modeling to explore the impact of sarcopenia on balance. In this study, we introduce a novel full-body MSK model comprising both the torso and lower limbs, with 290 muscle actuators controlling 23 degrees of freedom and supporting varying levels of sarcopenia. Using reinforcement learning coupled with curriculum learning and muscle synergy representations, we trained an agent to perform standing balance on a backward-sliding plate and compared its behavior to human experiments. Our results demonstrate that, without pre-recorded experimental data, both healthy and sarcopenic agents can reproduce ankle and hip balancing strategies consistent with experimental findings. Furthermore, we show that as the degree of sarcopenia increases, the agent adapts its balancing strategy based on the platform’s acceleration. The full code is open-sourced and can be found in this repository: https://github.com/cherylwang20/StandingBalance