{"title":"Hierarchical Control for Cooperative Teams in Competitive Autonomous Racing","authors":"Rishabh Saumil Thakkar;Aryaman Singh Samyal;David Fridovich-Keil;Zhe Xu;Ufuk Topcu","doi":"10.1109/TIV.2024.3363177","DOIUrl":"https://doi.org/10.1109/TIV.2024.3363177","url":null,"abstract":"We investigate the problem of autonomous racing among teams of cooperative agents that are subject to realistic racing rules. Our work extends previous research on hierarchical control in head-to-head autonomous racing by considering a generalized version of the problem while maintaining the two-level hierarchical control structure. A high-level tactical planner constructs a discrete game that encodes the complex rules using simplified dynamics to produce a sequence of target waypoints. The low-level path planner uses these waypoints as a reference trajectory and computes high-resolution control inputs by solving a simplified formulation of a racing game with a simplified representation of the realistic racing rules. We explore two approaches for the low-level path planner: training a multi-agent reinforcement learning (MARL) policy and solving a linear-quadratic Nash game (LQNG) approximation. We evaluate our controllers on simple and complex tracks against three baselines: an end-to-end MARL controller, a MARL controller tracking a fixed racing line, and an LQNG controller tracking a fixed racing line. Quantitative results show our hierarchical methods outperform the baselines in terms of race wins, overall team performance, and compliance with the rules. Qualitatively, we observe the hierarchical controllers mimic actions performed by expert human drivers such as coordinated overtaking, defending against multiple opponents, and long-term planning for delayed advantages.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 5","pages":"4845-4860"},"PeriodicalIF":14.0,"publicationDate":"2024-02-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141964746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Teleoperation Enhancement for Autonomous Vehicles Using Estimation Based Predictive Display","authors":"Gaurav Sharma;Rajesh Rajamani","doi":"10.1109/TIV.2024.3360410","DOIUrl":"https://doi.org/10.1109/TIV.2024.3360410","url":null,"abstract":"Teleoperation is increasingly used in the operation of delivery robots and is beginning to be utilized for certain autonomous vehicle intervention applications. This paper addresses the challenges in teleoperation of an autonomous vehicle due to latencies in wireless communication between the remote vehicle and the teleoperator station. Camera video images and Lidar data are typically delayed during wireless transmission but are critical for proper display of the remote vehicle's real-time road environment to the teleoperator. Data collected with experiments in this project show that a 0.5 second delay in real-time display makes it extremely difficult for the teleoperator to control the remote vehicle. This problem is addressed in the paper by using a predictive display (PD) system which provides intermediate updates of the remote vehicle's environment while waiting for actual camera images. The predictive display utilizes estimated positions of the ego vehicle and of other vehicles on the road computed using model-based extended Kalman filters. A crucial result presented in the paper is that vehicle motion models need to be inertial rather than relative and so tracking of other vehicles requires accurate localization of the ego vehicle itself. An experimental study using 5 human teleoperators is conducted to compare teleoperation performance with and without predictive display. A 0.5 second time-delay in camera images makes it impossible to control the vehicle to stay in its lane on curved roads, but the use of the developed predictive display system enables safe remote vehicle control with almost as accurate a performance as the delay-free case.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 3","pages":"4456-4469"},"PeriodicalIF":8.2,"publicationDate":"2024-02-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140820325","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Safety-Critical Parallel Trajectory Tracking Control of Maritime Autonomous Surface Ships Based on Integral Control Barrier Functions","authors":"Jiaxue Xu;Nan Gu;Dan Wang;Tieshan Li;Bing Han;Zhouhua Peng","doi":"10.1109/TIV.2024.3361477","DOIUrl":"https://doi.org/10.1109/TIV.2024.3361477","url":null,"abstract":"This article investigates the parallel trajectory tracking control of fully-actuated maritime autonomous surface ships (MASSs) in the presence of multiple stationary/moving ocean obstacles. A safety-critical parallel control architecture is proposed for the trajectory tracking control of MASSs. Specifically, an artificial MASS system is constructed based on a data-driven learning predictor where real-time and historical navigation data are both utilized to achieve the estimation of the unknown weights of Taylor polynomials and Fourier series. Then, a parallel trajectory tracking control law is designed based on the artificial system such that the MASS is capable of track the reference trajectory positively. Finally, integral control barrier functions are employed to encode input and safety constraints. A safety optimization signal is augmented to the designed parallel control law to achieve the collision avoidance of all ocean obstacles. Based on the stability and safety analyses, the tracking errors of the actual MASS system are verified to be uniformly ultimately bounded and the MASS system is safe. Numerical examples confirm the effectiveness of the designed safety-critical parallel trajectory tracking control scheme for the MASS.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 5","pages":"4979-4988"},"PeriodicalIF":14.0,"publicationDate":"2024-02-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141964760","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Share Your Preprint Research with the World!","authors":"","doi":"10.1109/TIV.2024.3394589","DOIUrl":"https://doi.org/10.1109/TIV.2024.3394589","url":null,"abstract":"","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 2","pages":"4232-4232"},"PeriodicalIF":8.2,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10510218","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140813984","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hongkai Yu;Xinyu Liu;Yonglin Tian;Yutong Wang;Chao Gou;Fei-Yue Wang
{"title":"Sora-Based Parallel Vision for Smart Sensing of Intelligent Vehicles: From Foundation Models to Foundation Intelligence","authors":"Hongkai Yu;Xinyu Liu;Yonglin Tian;Yutong Wang;Chao Gou;Fei-Yue Wang","doi":"10.1109/TIV.2024.3376575","DOIUrl":"https://doi.org/10.1109/TIV.2024.3376575","url":null,"abstract":"There are a large number of functional sensors installed on the modern intelligent vehicles. Many Artificial Intelligence based foundation models have been proposed for smart sensing to recognize the known object classes in the new but similar scenarios. However, it is still challenging for the foundation models of smart sensing to detect all the object classes in both seen and unseen scenarios. This letter aims at pushing the boundary of smart sensing research for intelligent vehicles. We first summarize the current widely-used foundation models and the foundation intelligence needed for smart sensing of intelligent vehicles. We then explain Sora-based Parallel Vision to boost the foundation models of smart sensing from basic intelligence (1.0) to enhanced intelligence (2.0) and final generalized intelligence (3.0). Several representative case studies are discussed to show the potential usages of Sora-based Parallel Vision, followed by its future research direction.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 2","pages":"3123-3126"},"PeriodicalIF":8.2,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140814020","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zhibin Shuai;Zheng Hu;Jiangtao Gai;Yijie Chen;Jicheng Chen;Hui Zhang;Fei-Yue Wang
{"title":"Metaverse-Enabled Intelligence for Open-Terrain Field Vehicle Fleets: Leveraging Parallel Intelligence and Edge Computing","authors":"Zhibin Shuai;Zheng Hu;Jiangtao Gai;Yijie Chen;Jicheng Chen;Hui Zhang;Fei-Yue Wang","doi":"10.1109/TIV.2024.3376461","DOIUrl":"https://doi.org/10.1109/TIV.2024.3376461","url":null,"abstract":"Open-terrain field vehicle (OTFV) fleets, including mining trucks, construction machinery, and agricultural machinery, often encounter significantly more intricate scenarios and unique challenges than road vehicles. Enhancing the intelligence level of OTFV fleets can significantly enhance their operational effectiveness and improve energy efficiency. This perspective paper introduces a metaverse-enabled framework to improve the intelligence levels of OTFV fleets. The metaverse-enabled framework consists of the parallel intelligence-based vehicle fleet control and edge computing-based vehicle dynamics control levels. We first delve into the framework's specifics, covering open-terrain field metaverse, parallel intelligence, edge computing, and human-vehicle cooperation. We further discuss critical issues such as artificial general intelligence (AGI) enabled large control models, vehicle teleoperation, communication privacy, and edge scenario engineering. Additionally, we provide a detailed account of edge computing and integrated domain control within the vehicle dynamics control level, illustrating the interactions among human drivers, domain controllers, vehicle systems and open-terrain field metaverse. Ultimately, the proposed framework can potentially empower intelligence to OTFV fleets and other equipment clusters with complicated system compositions and challenging missions in complex environments.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 2","pages":"3111-3116"},"PeriodicalIF":8.2,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140814069","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Intelligent Vehicles From Your HomePorts to Underwaters and Low Attitude Airspaces: SLAM for Smart Societies","authors":"Fei-Yue Wang","doi":"10.1109/TIV.2024.3373614","DOIUrl":"https://doi.org/10.1109/TIV.2024.3373614","url":null,"abstract":"This issue comprises 2 Perspectives, 2 Letters, and 82 Regular Papers. After \u0000<bold>Scanning the Issue</b>\u0000, I would like to initiate a broad discussion about Intelligent Vehicles (IVs), with an emphasis on smart logistics and autonomous mobility (SLAM) for smart societies.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 2","pages":"3092-3105"},"PeriodicalIF":8.2,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140814073","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}