A Five-Dimensional Three-Layer Digital Twin to Train a Reinforcement Learning Agent for Interaction Control of a Robotic Exoskeleton in Adolescent Idiopathic Scoliosis Rehabilitation

Adolescent idiopathic scoliosis (AIS) is a sideway curvature of the spinal column combined with a vertebral rotation that usually occurs in adolescents without any known causes. Bracing, the most common conservative treatment of AIS, has not fully exploited the benefits of the active control approaches powered by artificial intelligence (AI), although AI has entered a wide range of applications. The correction forces exerted by the brace are controlled passively by regulating the tightness of the brace's strap. Besides, training the learning-based control methods using a virtual model is of high importance in the AIS brace treatment, since training using trial and error on human subjects may result in unexpected pressure and injuries on the patient's torso. However, digital twin (DT) modeling, an emerging technology, has not been implemented into the AIS brace treatment yet. In this paper, reinforcement learning-based position-based impedance control (RLPIC) is proposed to enable a robotic brace to learn the desired physical interaction between the robotic brace and the human torso. A five-dimensional (5D) three-layer DT is also developed to be used for training the RLPIC in a simulated environment. The 5D three-layer DT consists of a physical system, a three-layer digital model of the physical system, including the robotic brace, human torso, and the physical human–robot interaction (HRI), a bidirectional connection between them, and an optimization dimension. A neural network-based regression model is also proposed to estimate the unknown parameters of the digital model. Numerical simulations and real-time experiments are performed to validate the 5D three-layer DT model. The proposed RLPIC trained using the 5D three-layer DT is verified using numerical simulations in terms of position tracking, velocity tracking, and HRI control. It is concluded that the proposed learning-based interaction control approach can improve the HRI control by learning the desired interaction in a simulated environment.