Learning Variable Impedance Control for Robotic Massage With Deep Reinforcement Learning: A Novel Learning Framework

Contact-rich manipulation tasks are difficult to program to be performed by robots. Traditional compliance control methods, such as impedance control, rely excessively on environmental models and are ineffective in the face of increasingly complex contact tasks. Reinforcement learning (RL) has now achieved great success in the fields of games and robotics. Autonomous learning of manipulation skills can empower robots with autonomous decision-making capabilities. To this end, this work introduces a novel learning framework that combines deep RL (DRL) and variable impedance control (VIC) to achieve robotic massage tasks. A skill policy is learned in joint space, which outputs the desired impedance gain and angle for each joint. To address the limitations of the sparse reward of DRL, an intrinsic curiosity module (ICM) was designed, which generates the intrinsic reward to encourage robots to explore more effectively. Simulation and real experiments were performed to verify the effectiveness of the proposed method. Our experiments demonstrate that contact-rich massage skills can be learned through the VIC–DRL framework based on the joint space in a simulation environment, and that the ICM can improve learning efficiency and overall performance in the task. Moreover, the generated policies have been demonstrated to still perform effectively on a real-world robot.