{"title":"基于分布式教练的蛇形机器人运动强化学习控制器","authors":"Yuanyuan Jia, Shugen Ma","doi":"10.1109/IROS47612.2022.9981749","DOIUrl":null,"url":null,"abstract":"Reinforcement learning commonly suffers from slow convergence speed and requires thousands of episodes, which makes it hard to be applied for physical robotic applications. Little research has been studied for snake robot control using RL because of the additional difficulty of high redundancy of freedom. Existing methods either adopts an asynchronous A3C structure or a joint state representation. We propose a distributed coach-based deep learning method for snake robot control, which can greatly expedite the training speed with less episodes. The major contributions include: 1) a completely distributed graphical formulation; 2) an explicit stochastic density propagation rule for each robot link; 3) various interaction models with uncertainty estimation. The preliminary results of both simulation and real-world experiments have demonstrated the promising performance in comparison with state-of-the-art.","PeriodicalId":431373,"journal":{"name":"2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)","volume":"41 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2022-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Distributed Coach-Based Reinforcement Learning Controller for Snake Robot Locomotion\",\"authors\":\"Yuanyuan Jia, Shugen Ma\",\"doi\":\"10.1109/IROS47612.2022.9981749\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Reinforcement learning commonly suffers from slow convergence speed and requires thousands of episodes, which makes it hard to be applied for physical robotic applications. Little research has been studied for snake robot control using RL because of the additional difficulty of high redundancy of freedom. Existing methods either adopts an asynchronous A3C structure or a joint state representation. We propose a distributed coach-based deep learning method for snake robot control, which can greatly expedite the training speed with less episodes. The major contributions include: 1) a completely distributed graphical formulation; 2) an explicit stochastic density propagation rule for each robot link; 3) various interaction models with uncertainty estimation. The preliminary results of both simulation and real-world experiments have demonstrated the promising performance in comparison with state-of-the-art.\",\"PeriodicalId\":431373,\"journal\":{\"name\":\"2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)\",\"volume\":\"41 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2022-10-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/IROS47612.2022.9981749\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/IROS47612.2022.9981749","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Distributed Coach-Based Reinforcement Learning Controller for Snake Robot Locomotion
Reinforcement learning commonly suffers from slow convergence speed and requires thousands of episodes, which makes it hard to be applied for physical robotic applications. Little research has been studied for snake robot control using RL because of the additional difficulty of high redundancy of freedom. Existing methods either adopts an asynchronous A3C structure or a joint state representation. We propose a distributed coach-based deep learning method for snake robot control, which can greatly expedite the training speed with less episodes. The major contributions include: 1) a completely distributed graphical formulation; 2) an explicit stochastic density propagation rule for each robot link; 3) various interaction models with uncertainty estimation. The preliminary results of both simulation and real-world experiments have demonstrated the promising performance in comparison with state-of-the-art.