{"title":"Task-Oriented Self-Imitation Learning for Robotic Autonomous Skill Acquisition","authors":"Chenyang Ran, Jianbo Su","doi":"10.1142/s0219843624500014","DOIUrl":"https://doi.org/10.1142/s0219843624500014","url":null,"abstract":"<p>The inferior sample efficiency of reinforcement learning (RL) and the requirement for high-quality demonstrations in imitation learning (IL) will hinder their application in real-world robots. To address this challenge, a novel self-evolution framework, named task-oriented self-imitation learning (TOSIL), is proposed. To circumvent external demonstrations, the top-K self-generated trajectories are chosen as expert data from both per-episode exploration and long-term return perspectives. Each transition is assigned a guide reward, which is formulated by these trajectories. The guide rewards update as the agent evolves, encouraging good exploration behaviors. This methodology guarantees that the agent explores in the direction relevant to the task, improving sample efficiency and asymptotic performance. The experimental results on locomotion and manipulation tasks indicate that the proposed framework outperforms other state-of-the-art RL methods. Furthermore, the integration of suboptimal trajectories has the potential to improve the sample efficiency while maintaining performance. This is a significant advancement in autonomous skill acquisition for robots.</p>","PeriodicalId":50319,"journal":{"name":"International Journal of Humanoid Robotics","volume":"21 1","pages":""},"PeriodicalIF":1.5,"publicationDate":"2024-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140153545","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Celebrating 20th Anniversary of IJHR","authors":"Ming Xie, John Weng, Giulio Sandini","doi":"10.1142/s021984362402002x","DOIUrl":"https://doi.org/10.1142/s021984362402002x","url":null,"abstract":"<p>In this editorial dedicated to the special issue which celebrates the 20th anniversary of the International Journal of Humanoid Robotics (IJHR), we outline the history of IJHR.</p>","PeriodicalId":50319,"journal":{"name":"International Journal of Humanoid Robotics","volume":"15 1","pages":""},"PeriodicalIF":1.5,"publicationDate":"2024-02-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140197941","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Anthology: Cognitive Developmental Humanoids Robotics","authors":"Minoru Asada","doi":"10.1142/s0219843624500026","DOIUrl":"https://doi.org/10.1142/s0219843624500026","url":null,"abstract":"<p>This paper explores the confluence of physical embodiment and social interaction in the context of Cognitive Developmental Humanoid Robotics (CDHR). By classifying varied interactions through developmental stages of the “self” and their interaction spheres, the discussion unearths profound insights into the composite nature of developmental processes. It presents a multi-dimensional exploration through different interaction scenarios, ranging from fetus-mother interactions to advanced large language models like ChatGPT, revealing the intrinsic connection between the physical and social realms of existence. In conclusion, this paper broadens the horizon of our understanding of the intricate interplays between physical embodiment and social interaction, setting the stage for more nuanced, ethically sound approaches and explorations in the realm of humanoid robotics and artificial intelligence.</p>","PeriodicalId":50319,"journal":{"name":"International Journal of Humanoid Robotics","volume":"9 1","pages":""},"PeriodicalIF":1.5,"publicationDate":"2024-02-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140197936","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Learning an Image-Based Visual Servoing Controller for Object Grasping","authors":"Shuaijun Wang, Lining Sun, Mantian Li, Pengfei Wang, Fusheng Zha, Wei Guo, Qiang Li","doi":"10.1142/s0219843623500330","DOIUrl":"https://doi.org/10.1142/s0219843623500330","url":null,"abstract":"<p>Adaptive and cooperative control of arms and fingers for natural object reaching and grasping, without explicit 3D geometric pose information, is observed in humans. In this study, an image-based visual servoing controller, inspired by human grasping behavior, is proposed for an arm-gripper system. A large-scale dataset is constructed using Pybullet simulation, comprising paired images and arm-gripper control signals mimicking expert grasping behavior. Leveraging this dataset, a network is directly trained to derive a control policy that maps images to cooperative grasp control. Subsequently, the learned synergy grasping policy from the network is directly applied to a real robot with the same configuration. Experimental results demonstrate the effectiveness of the algorithm. Videos can be found at https://www.bilibili.com/video/BV1tg4y1b7Qe/.</p>","PeriodicalId":50319,"journal":{"name":"International Journal of Humanoid Robotics","volume":"51 1","pages":""},"PeriodicalIF":1.5,"publicationDate":"2024-01-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140153453","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"ZMP - where are we after fifty-five years?","authors":"B. Borovac, M. Nikolic, M. Raković, S. Savic","doi":"10.1142/s0219843623500305","DOIUrl":"https://doi.org/10.1142/s0219843623500305","url":null,"abstract":"","PeriodicalId":50319,"journal":{"name":"International Journal of Humanoid Robotics","volume":"173 ","pages":""},"PeriodicalIF":1.5,"publicationDate":"2023-12-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139010688","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"From embodiment to super-embodiment: an approach to open-ended and human aligned intelligence/mind","authors":"Yasuo Kuniyoshi","doi":"10.1142/s0219843623500299","DOIUrl":"https://doi.org/10.1142/s0219843623500299","url":null,"abstract":"","PeriodicalId":50319,"journal":{"name":"International Journal of Humanoid Robotics","volume":" 34","pages":""},"PeriodicalIF":1.5,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138616351","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Masahiro Fujita, Yasunori Kawanami, Kiyokazu Miyazawa, Masaya Kinoshita, Kunihito Sawai, F. Yamasaki, Tatsuya Matsui, Ken Endo, Shu Ishiguro, Hiroaki Kitano
{"title":"Stories of qrio and pino, and beyond: lessons learned from small humanoid projects from R&D to business","authors":"Masahiro Fujita, Yasunori Kawanami, Kiyokazu Miyazawa, Masaya Kinoshita, Kunihito Sawai, F. Yamasaki, Tatsuya Matsui, Ken Endo, Shu Ishiguro, Hiroaki Kitano","doi":"10.1142/s0219843623500275","DOIUrl":"https://doi.org/10.1142/s0219843623500275","url":null,"abstract":"","PeriodicalId":50319,"journal":{"name":"International Journal of Humanoid Robotics","volume":"52 3","pages":""},"PeriodicalIF":1.5,"publicationDate":"2023-11-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139264024","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}