A physics-informed train on synthetic and test on real method for evaluating large language model-generated safety-critical traffic scenarios

Abstract

Corner cases, which are rare and high-risk scenarios such as safety-critical behaviors in autonomous vehicle operations, present significant modeling challenges due to their low occurrence probability and limited data availability. Large language models (LLMs) offer new potential for synthesizing such scenarios, but existing evaluation metrics are inadequate because corner case data typically lack one-to-one mapping to real samples and have extremely limited instances. To address this, we propose a two-stage evaluation framework, that is, a physics-informed train on synthetic and test on real (PI-TSTR) framework. Using safety-critical car-following (CF) scenarios as an example, we design a prompting and interpolation strategy to guide LLMs in generating physically feasible synthetic follower trajectories from real leading vehicle inputs. We then evaluate the generated data by training several CF models, that is, extended S-shaped three-parameter (ES3) model, Gipps model, optimal velocity model (OVM), improved full velocity difference model (IFVDM), intelligent driver model (IDM), and testing their performances on real-world data. The CF models trained on LLM-generated trajectories show strong generalization to real scenarios, validating the quality of the synthetic data. This framework provides a physics-grounded approach for evaluating LLM-generated data in safety-critical, data-scarce domains.