Knowledge guided controllable diffusion for enhanced autonomous driving scenarios generation

Considering the shortcomings of the natural driving datasets composed by typical scenarios in the verification and evaluation of autonomous driving, how to improve the diversity, realism and controllability to satisfy the complex driving scenarios need is an urgent problem to be solved. Therefore, a knowledge-guided controllable diffusion (KGCD) scenarios generation framework is proposed, which allows for the generation behaviors that conform to expected attributes through a differentiable cost designed based on prior driving knowledge during the inference stage. Specifically, factorized attention is utilized for capturing and fusion of the spatio-temporal interaction features for the scene embeddings, based on which the accurate scene-level interactive joint motion prediction is achieved. In the inference stage, diversified guidance is designed based on prior knowledge and embedded into a generation framework to guide the generation of controllable and realistic driving scenarios, filling the gap in the existing scenario library. In response to the slow inference speed of typical diffusion model, a learnable motion pattern estimator (MPE) module is proposed to improve inference speed while ensuring generation quality. Based on the experiments with different datasets, KGCD proposed in this paper can satisfy the requirements of diverse scene generation. Compared with the best performance obtained by other baseline, the realism, controllability and stability metrics have been improved on average by 3.57%, 7.66% and 7.71% respectively evaluated by nuScenes dataset.