Fuzzy Q-Learning interaction controller design for collaborative robot

Background: In physical human-robot interaction (pHRI), admittance control is widely used. The most critical thing in admittance control is the configuration of admittance parameters, but a constant admittance value can not meet the needs of interactive indicators smoothness especially. Variable admittance control is a method to overcome this limitation by adjusting the admittance value in real time. This paper proposes a fuzzy Q-learning (FQL) variable admittance control system, which integrates the fuzzy system (FIS) and reinforcement learning method Q-learning.  Methods: FIS is used to turn a continuous input state into fuzzy set and Q-learning is used to train the premise strength of fuzzy rules to get the optimal policy of variable admittance value. To verify the performance of this method, an experiment was performed using an AUBO i5 robot. Training trajectory is point-to-point (PTP) trajectory, several interaction variables before and after training by the algorithm are compared to show the validity of algorithm. Results: Experimental results show that the reward converges to a smaller value in about 25 episodes, and the reward of the last five episodes reduces by 68%. The motion trajectory after algorithm training is closer to the ideal min-jerk trajectory and the deviation and mean value of interaction force become smaller. Conclusions: The proposed FQL method can converge in a few episodes and can improve the performance of pHRI by minimizing the jerk based cost function