VistaRAG: Toward Safe and Trustworthy Autonomous Driving Through Retrieval-Augmented Generation

IF 14 1区工程技术 Q1 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

IEEE Transactions on Intelligent Vehicles Pub Date : 2024-04-01 DOI:10.1109/TIV.2024.3396450

Xingyuan Dai;Chao Guo;Yun Tang;Haichuan Li;Yutong Wang;Jun Huang;Yonglin Tian;Xin Xia;Yisheng Lv;Fei-Yue Wang

{"title":"VistaRAG: Toward Safe and Trustworthy Autonomous Driving Through Retrieval-Augmented Generation","authors":"Xingyuan Dai;Chao Guo;Yun Tang;Haichuan Li;Yutong Wang;Jun Huang;Yonglin Tian;Xin Xia;Yisheng Lv;Fei-Yue Wang","doi":"10.1109/TIV.2024.3396450","DOIUrl":null,"url":null,"abstract":"Autonomous driving based on foundation models has recently garnered widespread attention. However, the risk of hallucinations inherent in foundation models could compromise the safety and reliability of autonomous driving systems. This letter, as part of a series of reports from the Distributed/Decentralized Hybrid Workshop on Foundation/Infrastructure Intelligence (DHW-FII), aims to tackle these issues. We introduce VistaRAG, which integrates retrieval-augmented generation (RAG) technologies into autonomous driving systems based on foundation models, to address the inherent reliability challenges in decision-making. VistaRAG employs a dynamic retrieval mechanism to access highly relevant driving experience, real-time road network status, and other contextual information from external databases. This aids foundation models in informed reasoning and decision-making, thereby enhancing the safety and trustworthiness of foundation-model-based autonomous driving systems under complex traffic scenarios.","PeriodicalId":36532,"journal":{"name":"IEEE Transactions on Intelligent Vehicles","volume":"9 4","pages":"4579-4582"},"PeriodicalIF":14.0000,"publicationDate":"2024-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Intelligent Vehicles","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/10518077/","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}

引用次数: 0

Abstract

Autonomous driving based on foundation models has recently garnered widespread attention. However, the risk of hallucinations inherent in foundation models could compromise the safety and reliability of autonomous driving systems. This letter, as part of a series of reports from the Distributed/Decentralized Hybrid Workshop on Foundation/Infrastructure Intelligence (DHW-FII), aims to tackle these issues. We introduce VistaRAG, which integrates retrieval-augmented generation (RAG) technologies into autonomous driving systems based on foundation models, to address the inherent reliability challenges in decision-making. VistaRAG employs a dynamic retrieval mechanism to access highly relevant driving experience, real-time road network status, and other contextual information from external databases. This aids foundation models in informed reasoning and decision-making, thereby enhancing the safety and trustworthiness of foundation-model-based autonomous driving systems under complex traffic scenarios.

查看原文本刊更多论文

VistaRAG：通过检索增强生成实现安全可信的自动驾驶

基于基础模型的自动驾驶最近受到广泛关注。然而，基础模型固有的幻觉风险可能会损害自动驾驶系统的安全性和可靠性。作为基础/基础设施智能分布式/分散式混合研讨会（DHW-FII）系列报告的一部分，这封信旨在解决这些问题。我们介绍了 VistaRAG，它将检索增强生成（RAG）技术集成到基于地基模型的自动驾驶系统中，以解决决策中固有的可靠性挑战。VistaRAG 采用动态检索机制，从外部数据库获取高度相关的驾驶经验、实时路网状态和其他上下文信息。这有助于基础模型进行知情推理和决策，从而提高基于基础模型的自动驾驶系统在复杂交通场景下的安全性和可信度。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

IEEE Transactions on Intelligent Vehicles Mathematics-Control and Optimization

CiteScore

12.10

自引率

13.40%

发文量

177

期刊介绍： The IEEE Transactions on Intelligent Vehicles (T-IV) is a premier platform for publishing peer-reviewed articles that present innovative research concepts, application results, significant theoretical findings, and application case studies in the field of intelligent vehicles. With a particular emphasis on automated vehicles within roadway environments, T-IV aims to raise awareness of pressing research and application challenges. Our focus is on providing critical information to the intelligent vehicle community, serving as a dissemination vehicle for IEEE ITS Society members and others interested in learning about the state-of-the-art developments and progress in research and applications related to intelligent vehicles. Join us in advancing knowledge and innovation in this dynamic field.