SofaGym:基于软机器人仿真的开放式强化学习平台。

IF 6.4 2区 计算机科学 Q1 ROBOTICS
Pierre Schegg, Etienne Ménager, Elie Khairallah, Damien Marchal, Jérémie Dequidt, Philippe Preux, Christian Duriez
{"title":"SofaGym:基于软机器人仿真的开放式强化学习平台。","authors":"Pierre Schegg,&nbsp;Etienne Ménager,&nbsp;Elie Khairallah,&nbsp;Damien Marchal,&nbsp;Jérémie Dequidt,&nbsp;Philippe Preux,&nbsp;Christian Duriez","doi":"10.1089/soro.2021.0123","DOIUrl":null,"url":null,"abstract":"<p><p>OpenAI Gym is one of the standard interfaces used to train Reinforcement Learning (RL) Algorithms. The Simulation Open Framework Architecture (SOFA) is a physics-based engine that is used for soft robotics simulation and control based on real-time models of deformation. The aim of this article is to present <i>SofaGym</i>, an open-source software to create OpenAI Gym interfaces, called environments, out of soft robot digital twins. The link between soft robotics and RL offers new challenges for both fields: representation of the soft robot in an RL context, complex interactions with the environment, use of specific mechanical tools to control soft robots, transfer of policies learned in simulation to the real world, etc. The article presents the large possible uses of SofaGym to tackle these challenges by using RL and planning algorithms. This publication contains neither new algorithms nor new models but proposes a new platform, open to the community, that offers non existing possibilities of coupling RL to physics-based simulation of soft robots. We present 11 environments, representing a wide variety of soft robots and applications; we highlight the challenges showcased by each environment. We propose methods of solving the task using traditional control, RL, and planning and point out research perspectives using the platform.</p>","PeriodicalId":48685,"journal":{"name":"Soft Robotics","volume":"10 2","pages":"410-430"},"PeriodicalIF":6.4000,"publicationDate":"2023-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"10","resultStr":"{\"title\":\"SofaGym: An Open Platform for Reinforcement Learning Based on Soft Robot Simulations.\",\"authors\":\"Pierre Schegg,&nbsp;Etienne Ménager,&nbsp;Elie Khairallah,&nbsp;Damien Marchal,&nbsp;Jérémie Dequidt,&nbsp;Philippe Preux,&nbsp;Christian Duriez\",\"doi\":\"10.1089/soro.2021.0123\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>OpenAI Gym is one of the standard interfaces used to train Reinforcement Learning (RL) Algorithms. The Simulation Open Framework Architecture (SOFA) is a physics-based engine that is used for soft robotics simulation and control based on real-time models of deformation. The aim of this article is to present <i>SofaGym</i>, an open-source software to create OpenAI Gym interfaces, called environments, out of soft robot digital twins. The link between soft robotics and RL offers new challenges for both fields: representation of the soft robot in an RL context, complex interactions with the environment, use of specific mechanical tools to control soft robots, transfer of policies learned in simulation to the real world, etc. The article presents the large possible uses of SofaGym to tackle these challenges by using RL and planning algorithms. This publication contains neither new algorithms nor new models but proposes a new platform, open to the community, that offers non existing possibilities of coupling RL to physics-based simulation of soft robots. We present 11 environments, representing a wide variety of soft robots and applications; we highlight the challenges showcased by each environment. We propose methods of solving the task using traditional control, RL, and planning and point out research perspectives using the platform.</p>\",\"PeriodicalId\":48685,\"journal\":{\"name\":\"Soft Robotics\",\"volume\":\"10 2\",\"pages\":\"410-430\"},\"PeriodicalIF\":6.4000,\"publicationDate\":\"2023-04-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"10\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Soft Robotics\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://doi.org/10.1089/soro.2021.0123\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ROBOTICS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Soft Robotics","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1089/soro.2021.0123","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ROBOTICS","Score":null,"Total":0}
引用次数: 10

摘要

OpenAI Gym是用于训练强化学习(RL)算法的标准接口之一。仿真开放框架体系结构(SOFA)是一个基于物理的引擎,用于基于实时变形模型的软机器人仿真和控制。本文的目的是介绍SofaGym,这是一个开源软件,用于创建OpenAI Gym接口,称为环境,出自软机器人数字双胞胎。软机器人和强化学习之间的联系为这两个领域提供了新的挑战:在强化学习环境中表示软机器人,与环境的复杂交互,使用特定的机械工具来控制软机器人,将模拟中学到的策略转移到现实世界等。本文介绍了SofaGym通过使用强化学习和规划算法来解决这些挑战的大量可能用途。本出版物既不包含新算法也不包含新模型,但提出了一个向社区开放的新平台,该平台提供了将RL耦合到基于物理的软机器人仿真的不存在的可能性。我们展示了11个环境,代表了各种各样的软机器人和应用;我们强调了每种环境所带来的挑战。我们提出了使用传统控制、强化学习和计划来解决任务的方法,并指出了使用该平台的研究前景。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
SofaGym: An Open Platform for Reinforcement Learning Based on Soft Robot Simulations.

OpenAI Gym is one of the standard interfaces used to train Reinforcement Learning (RL) Algorithms. The Simulation Open Framework Architecture (SOFA) is a physics-based engine that is used for soft robotics simulation and control based on real-time models of deformation. The aim of this article is to present SofaGym, an open-source software to create OpenAI Gym interfaces, called environments, out of soft robot digital twins. The link between soft robotics and RL offers new challenges for both fields: representation of the soft robot in an RL context, complex interactions with the environment, use of specific mechanical tools to control soft robots, transfer of policies learned in simulation to the real world, etc. The article presents the large possible uses of SofaGym to tackle these challenges by using RL and planning algorithms. This publication contains neither new algorithms nor new models but proposes a new platform, open to the community, that offers non existing possibilities of coupling RL to physics-based simulation of soft robots. We present 11 environments, representing a wide variety of soft robots and applications; we highlight the challenges showcased by each environment. We propose methods of solving the task using traditional control, RL, and planning and point out research perspectives using the platform.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Soft Robotics
Soft Robotics ROBOTICS-
CiteScore
15.50
自引率
5.10%
发文量
128
期刊介绍: Soft Robotics (SoRo) stands as a premier robotics journal, showcasing top-tier, peer-reviewed research on the forefront of soft and deformable robotics. Encompassing flexible electronics, materials science, computer science, and biomechanics, it pioneers breakthroughs in robotic technology capable of safe interaction with living systems and navigating complex environments, natural or human-made. With a multidisciplinary approach, SoRo integrates advancements in biomedical engineering, biomechanics, mathematical modeling, biopolymer chemistry, computer science, and tissue engineering, offering comprehensive insights into constructing adaptable devices that can undergo significant changes in shape and size. This transformative technology finds critical applications in surgery, assistive healthcare devices, emergency search and rescue, space instrument repair, mine detection, and beyond.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信