Acquisition of Flipper Motion in Step-Climbing of Tracked Robot Using Reinforcement Learning

Remotely piloted robots have been expected for disaster operations to prevent secondary disasters to rescuers. The robots are required to have a high performance to overcome obstacles such as debris and bumps. Tracked robots with flipper arms on the front, back, left, and right sides can improve their ability to overcome bumps by changing the flipper angles, and are thus expected to be used as rescue robots. However, the six degrees of freedom of the left and right crawlers and four flipper arms require a high level of skill to maneuver the robot. Therefore, a semi-autonomous control system that automatically controls the flipper arms according to the terrain is expected. In this study, we proposed a method to determine the front and rear flipper angles in step-climbing using reinforcement learning with Double Deep Q Network. The input data were the step height, distance to the step, and current front and rear flipper angles. The outputs were amounts of front and rear flipper angle variations. The behavior of the robot and rewards were calculated using a quasi-static model that considers the slips between the step, floor, and crawler. Positive rewards were given for successful step stepping over steps and negative rewards for unsuccessful steps. Furthermore, the flipper motions at which slippage decreased were obtained by subtracting the sum of the squared values of the slippage rates from the rewards. As the results, it was confirmed that the robot slips less when the body is lifted by the rear flipper than when it runs over along the step shape.