{"title":"Solving large-scale multi-agent tasks via transfer learning with dynamic state representation","authors":"Lintao Dou, Zhen Jia, Jian Huang","doi":"10.1177/17298806231162440","DOIUrl":"https://doi.org/10.1177/17298806231162440","url":null,"abstract":"Many research results have emerged in the past decade regarding multi-agent reinforcement learning. These include the successful application of asynchronous advantage actor-critic, double deep Q-network and other algorithms in multi-agent environments, and the more representative multi-agent training method based on the classical centralized training distributed execution algorithm QMIX. However, in a large-scale multi-agent environment, training becomes a major challenge due to the exponential growth of the state-action space. In this article, we design a training scheme from small-scale multi-agent training to large-scale multi-agent training. We use the transfer learning method to enable the training of large-scale agents to use the knowledge accumulated by training small-scale agents. We achieve policy transfer between tasks with different numbers of agents by designing a new dynamic state representation network, which uses a self-attention mechanism to capture and represent the local observations of agents. The dynamic state representation network makes it possible to expand the policy model from a few agents (4 agents, 10 agents) task to large-scale agents (16 agents, 50 agents) task. Furthermore, we conducted experiments in the famous real-time strategy game Starcraft II and the multi-agent research platform MAgent. And also set unmanned aerial vehicles trajectory planning simulations. Experimental results show that our approach not only reduces the time consumption of a large number of agent training tasks but also improves the final training performance.","PeriodicalId":50343,"journal":{"name":"International Journal of Advanced Robotic Systems","volume":" ","pages":""},"PeriodicalIF":2.3,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47450931","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Feature extraction and matching combined with depth information in visual simultaneous localization and mapping","authors":"Yunpeng Sun, Xiaoli Li","doi":"10.1177/17298806231158298","DOIUrl":"https://doi.org/10.1177/17298806231158298","url":null,"abstract":"Estimating the camera trajectories is very important for the performance of visual simultaneous localization and mapping. However, visual simultaneous localization and mapping systems based on RGB image are generally not robust in complex situations such as low-textures or large illumination variations. In order to solve this problem, more environmental information is added by introducing depth information, and a feature extraction and matching algorithm combining depth information is proposed. In this article, firstly, the intrinsic mechanism that depth image is used to extract and match feature points is discussed. Then depth information and appearance information are comprehensively considered to extract and describe feature points. Finally, the matching problem of feature points is transformed into a regression and classification problem, with which a matching model is presented in a data-driven way. Experimental results show that our algorithm has better distribution uniformity and matching accuracy and can effectively improve the trajectory accuracy and drift degree of the simultaneous localization and mapping system.","PeriodicalId":50343,"journal":{"name":"International Journal of Advanced Robotic Systems","volume":" ","pages":""},"PeriodicalIF":2.3,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48753751","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Visual application of navigation framework in cyber-physical system for mobile robot to prevent disease","authors":"Thanh Phuong Nguyen, Hung Nguyen, H. Ngo","doi":"10.1177/17298806231162202","DOIUrl":"https://doi.org/10.1177/17298806231162202","url":null,"abstract":"In this article, we propose the visual application of a navigation framework for a wheeled robot to disinfect surfaces. Since dynamic environments are complicated, advanced sensors are integrated into the hardware platform to enhance the navigation task. The 2D lidar UTM-30LX from Hokuyo attached to the front of the robot can cover a wide scanning area. To provide better results in laser scan matching, an inertial measurement unit was integrated into the robot’s body. The output of this combination feeds into a global costmap for monitoring and navigation. Additionally, incremental encoders that obtain high-resolution position data are connected to the rear wheels. The role of the positioning sensor is to identify the existing location of the robot in a local costmap. To detect the appearance of a human, a Kinect digital camera is fixed to the top of the robot. All feedback signals are combined in the host computer to navigate the autonomous robot. For disinfection missions, the robot must carry several ultraviolet lamps to autonomously patrol in unknown environments. To visualize the robot’s effectiveness, our approach was validated using both a virtual simulation and an experimental test. The contributions of this work are summarized as follows: (i) a structure for ultraviolet-based hardware was first established; (ii) the theoretical computations for the robot’s localization in the 3D workspace will play a fundamental role in further developments; and (iii) data fusion from advanced sensing devices was integrated to enable navigation in uncertain environments.","PeriodicalId":50343,"journal":{"name":"International Journal of Advanced Robotic Systems","volume":" ","pages":""},"PeriodicalIF":2.3,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45618668","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xian Xu, Meijia Wang, Yanfeng Zheng, Chunlin Zhou, Yaozhi Luo
{"title":"An improved numerical model for locomotive tensegrity systems based on vector form intrinsic finite element","authors":"Xian Xu, Meijia Wang, Yanfeng Zheng, Chunlin Zhou, Yaozhi Luo","doi":"10.1177/17298806231162442","DOIUrl":"https://doi.org/10.1177/17298806231162442","url":null,"abstract":"Locomotive robot based on tensegrity has recently drawn much attention due to its lightweight and flexibility. This article presents an improved numerical model for locomotive tensegrities. The previously used bar element for struts is replaced by beam element, and rigid joint element is used to consider more details of the tendon–strut connections. The vector form intrinsic finite element (VFIFE) method is adopted to formulate the numerical model and carry out the simulation. The improvement of the proposed model on the prediction of feasible rolling gaits is quantitatively verified by experiments on a six-strut locomotive tensegrity. Mann–Whitney U test is adopted, and the p value between the experimental success rates of the gait primitives generated by the improved model and the rates of the gait primitives generated by the previous model is 1.46 × 10 − 12 . It is shown that the improved model is more consistent with the experiment by considering the details of the tendon–strut connection.","PeriodicalId":50343,"journal":{"name":"International Journal of Advanced Robotic Systems","volume":" ","pages":""},"PeriodicalIF":2.3,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46582592","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"High-definition 3D map for autonomous vehicles in Vietnam: A comparison between graph simultaneous localization and mapping and the normal distributions transform algorithm","authors":"P. T. Nam, Le Anh Son, H. Nang","doi":"10.1177/17298806231157551","DOIUrl":"https://doi.org/10.1177/17298806231157551","url":null,"abstract":"Autonomous vehicles have a fundamental part to play in future transportation. In order to work, such vehicles need a way of determining their exact location. To date, two techniques have proved applicable: geolocation using GPS and 3D mapping. In the latter case, the creation of the 3D map is an essential step in preparing the software. This manuscript presents a novel method for creating a high-definition 3D map using a Velodyne sensor. The primary objective is to achieve deeper understanding of the map creation algorithm and to lay the groundwork for future autonomous vehicle architecture. In this manuscript, two high-definition mapping algorithms are applied: the graph simultaneous localization and mapping algorithm and the normal distributions transform algorithm. The results of both displayed different strengths and weaknesses upon creating a high-definition 3D map, which may help other researchers to select the best algorithm in future.","PeriodicalId":50343,"journal":{"name":"International Journal of Advanced Robotic Systems","volume":" ","pages":""},"PeriodicalIF":2.3,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42174536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yixu Wang, Yifan Liu, Xiaoqing Guan, Tao Hu, Ziang Zhang, You Wang, Jie Hao, Guang Li
{"title":"Robust servo linear quadratic regulator controller based on state compensation and velocity feedforward of the spherical robot: Theory and experimental verification","authors":"Yixu Wang, Yifan Liu, Xiaoqing Guan, Tao Hu, Ziang Zhang, You Wang, Jie Hao, Guang Li","doi":"10.1177/17298806231153229","DOIUrl":"https://doi.org/10.1177/17298806231153229","url":null,"abstract":"There are few studies on the lateral motion of spherical robots. In this article, a new algorithm is proposed to solve the problem of low control accuracy of the lateral motion. A single lateral motion model is established, and the optimal solution of the linear quadratic regulator in infinite time domain is obtained. Aiming at the problems of longitudinal velocity and lateral angle coupling and low control precision, state compensation and velocity feedforward are carried out, and an improved robust servo linear quadratic regulator control algorithm is proposed. Experiments show that the proposed lateral control algorithm has strong adaptability and robustness to changing speeds and lateral angles, and the control effect is stable and reliable.","PeriodicalId":50343,"journal":{"name":"International Journal of Advanced Robotic Systems","volume":" ","pages":""},"PeriodicalIF":2.3,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47309273","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dongxia Wang, Liangwen Wang, Hongchang Xie, Zhenzhen Wu, G. Xie, Caidong Wang, Huadong Zheng, Zhuang Wang
{"title":"Configuration design and grasping contact accuracy analysis of a four-finger manipulator used for grasping fragile objects of elliptic cross section","authors":"Dongxia Wang, Liangwen Wang, Hongchang Xie, Zhenzhen Wu, G. Xie, Caidong Wang, Huadong Zheng, Zhuang Wang","doi":"10.1177/17298806231157339","DOIUrl":"https://doi.org/10.1177/17298806231157339","url":null,"abstract":"A type of manipulator configuration with four-fingers is put forward, for grasping fragile hollow workpieces, with adjustable finger length and finger pad shape. Within a certain range, the manipulator can be used to grasp the internal cross section of fragile objects with different size of circular and oval shape. The design idea and structure of the manipulator are introduced, while the join positions between the finger pad and the finger body, as determined by uniform interpolation or Chebyshev interpolation, are comparatively analyzed and researched regarding forming force contact with workpiece. During the process of grasping, the internal forces and deformations between finger pad and workpiece are analyzed, based on a constructed finite element analysis model. The calculation example shows that, under the same grasping parameters, the maximum impact force on the workpiece is reduced by 63%, when the curvature adjustment points for the finger pad are distributed according to the Chebyshev interpolation, compared to their equal spacing distribution. Research has provided a theoretical basis for the design optimization of the finger pad structure and the connection point positions. For using manipulator to grasp objects with different size of circular and oval shape, the working space of the proposed manipulator is studied. The experiments show that, the manipulator structure, as presented in this article, can meet the requirements of relevant tasks.","PeriodicalId":50343,"journal":{"name":"International Journal of Advanced Robotic Systems","volume":" ","pages":""},"PeriodicalIF":2.3,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42729984","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xueyu Du, Maosong Wang, Wen-qi Wu, Pei-yuan Zhou, Jia-rui Cui
{"title":"State transformation extended Kalman filter–based tightly coupled strapdown inertial navigation system/global navigation satellite system/laser Doppler velocimeter integration for seamless navigation of unmanned ground vehicle in urban areas","authors":"Xueyu Du, Maosong Wang, Wen-qi Wu, Pei-yuan Zhou, Jia-rui Cui","doi":"10.1177/17298806231158462","DOIUrl":"https://doi.org/10.1177/17298806231158462","url":null,"abstract":"With the rapid development of unmanned ground vehicle industry, how to achieve continuous, reliable, and high-accuracy navigation becomes very important. At present, the integrated navigation with global navigation satellite system and strapdown inertial navigation system is the most mature and effective navigation technology for unmanned ground vehicle. However, this technique depends on the signal accuracy of global navigation satellite system. When the receiver cannot capture four or more satellite signals for a long time or the satellite completely invalid, it cannot provide accurate navigation and positioning information for the unmanned ground vehicle. Therefore, this article combine the observation information of strapdown inertial navigation system, global navigation satellite system, and laser Doppler velocimeter to propose a high-precision seamless navigation technique of unmanned ground vehicle based on state transformation extended Kalman filter. Under different land vehicle driving environments and global navigation satellite system signal quality conditions, the seamless navigation technique is evaluated through global navigation satellite system interruption simulation and land vehicle experiments. The experimental results show that the strapdown inertial navigation system/global navigation satellite system/laser Doppler velocimeter tightly coupled integration seamless navigation has good environmental adaptability and reliability and can maintain high navigation accuracy under high frequency global navigation satellite system–signal blockage conditions in urban areas.","PeriodicalId":50343,"journal":{"name":"International Journal of Advanced Robotic Systems","volume":" ","pages":""},"PeriodicalIF":2.3,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41342604","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Universal nonlinear disturbance observer for robotic manipulators","authors":"Feilong Zhang, Xin Zhang, Qingxin Li, Hualiang Zhang","doi":"10.1177/17298806231167669","DOIUrl":"https://doi.org/10.1177/17298806231167669","url":null,"abstract":"Dynamic uncertainties and unknown disturbances will degrade the tracking performance of robots. When using a disturber observer-based controller, the key to effectively compensate for these uncertainties is to measure or estimate as accurately as possible any disturbance. To relax restrictions on the design of the current nonlinear disturbance observer for the robot, a universal nonlinear disturbance observer is proposed for higher estimation performance. The stability of the proposed universal nonlinear disturbance observer is theoretically analyzed and the boundaries of estimation error are proven according to the vector differential equation. Finally, the proposed universal nonlinear disturbance observer is evaluated via simulation by comparison to the nonlinear disturbance observer. The result shows a faster estimation speed and a higher estimation accuracy of the universal nonlinear disturbance observer.","PeriodicalId":50343,"journal":{"name":"International Journal of Advanced Robotic Systems","volume":" ","pages":""},"PeriodicalIF":2.3,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42722809","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Minggang Li, Hanxu Sun, Long Ma, Panpan Gao, Dongshui Huo, Zhantong Wang, Ping Sun
{"title":"Special spherical mobile robot for planetary surface exploration: A review","authors":"Minggang Li, Hanxu Sun, Long Ma, Panpan Gao, Dongshui Huo, Zhantong Wang, Ping Sun","doi":"10.1177/17298806231162207","DOIUrl":"https://doi.org/10.1177/17298806231162207","url":null,"abstract":"Considering the requirements of high scientific return, low cost, less complexity, and more reliability for the robot proposed by the extreme environment exploration task on the planet surface, this article comprehensively reviews the history of the special spherical robot used for extraterrestrial surface exploration and summarizes the environmental characteristics and task difficulties of different planet surface. This article compares the advantages of different types of ground spherical robots and points out the superiority of special spherical robots, such as omni-direction, airtightness, zero-radius turning, under-actuated, swarming, and lightweight. In addition, the research progress of special spherical robots for extraterrestrial exploration, such as wind ball, jumping ball, fly ball, ball with leg, pendulum driven ball, tensegrity structure, are reviewed respectively. Finally, the performance characteristics of all these robots are analyzed, their application scope given.","PeriodicalId":50343,"journal":{"name":"International Journal of Advanced Robotic Systems","volume":" ","pages":""},"PeriodicalIF":2.3,"publicationDate":"2023-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49543895","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}