Large language model based system with causal inference and Chain-of-Thoughts reasoning for traffic scene risk assessment

Evaluating potential traffic scene risks is crucial for the decision-making process in autonomous driving systems. Large Language Models (LLMs), due to their advanced scene understanding and reasoning capabilities, offer a new paradigm for decision-making in autonomous driving. However, when used to evaluate traffic scene risks, LLM faces challenges of model hallucination and slow inference speed. Therefore, we have developed a TrafficRiskGPT-based system. Initially, we designed the TrafficRiskGPT model, which is a large language model specifically used for traffic scene risk reasoning. Its core is based on the LLaMA3-8B model, and it incorporates a large amount of traffic risk datasets for LoRA (Low-Rank Adaptation) fine-tuning, enabling the model to deeply understand traffic scene risks. On this basis, we designed a system around the TrafficRiskGPT model. The system establishes a knowledge base based on traffic scene risks and incorporated HNSW (Hierarchical Navigable Small World graphs) method to improve the retrieval efficiency of the knowledge base, introduces vLLM technology to improve the system’s inference speed, and constructs a comprehensive risk evaluation metric to assess the system’s performance on traffic scene risks. Finally, we designed a CI-CoT (Causal Inference Chain-of-Thought) technique, allowing the system to gradually evaluate the traffic risks associated with each decision, thereby reducing model hallucinations and slow inference speed issues. Our experiments show that in the same scenarios, in terms of vehicle collision rates, our method reduces the rate by 7.3% compared to GPT4o-mini, significantly reduces outputs irrelevant to traffic scene risks, and improves model inference speed by a factor of 3.62.