Autonomous control of soft robots using safe reinforcement learning and covariance matrix adaptation

The control of soft robots (such as continuum robots) poses significant challenges due to their coupled dynamics with significant inherent nonlinearities. Recently, model-free reinforcement learning algorithms have been proposed as an attractive alternative to model-based methods to address such a challenging control problem through unsupervised learning. However, the safety of robots is usually ignored while training such algorithms. This is particularly important for medical applications of soft robots. Also, the curse of dimensionality in soft robots makes it difficult for a reinforcement learning algorithm to develop an optimal controller. In this work, we propose a safe phasic soft actor–critic algorithm with a covariance matrix adaptation network which is then tested on different soft robots. We demonstrate that the proposed algorithm could learn an optimal policy quickly while satisfying the safety constraints. We formulated and tested our algorithm for (i) multigait soft robot; (ii) soft gripper robot; and (iii) soft robotic trunk. The proposed algorithm achieved an average of 150% higher rewards compared to other state-of-the-art algorithms. Also, adding the safety layer helped reduce the tracking error by 8 times when compared to the algorithm without a safety layer. The policy is validated in Simulation Open Framework Architecture (SOFA) simulations against other state-of-the-art algorithms in terms of tracking errors.