Adopting Graph Neural Networks to Understand and Reason About Dynamic Driving Scenarios

With advances in Deep Neural Networks (DNN), Automated Driving Systems (ADS) enable the vehicle to perceive their surroundings in dynamic driving scenarios and perform behaviors by collecting operational data from sensors such as LiDAR and cameras. Current DNN typically detect objects by analyzing and classifying unstructured data (e.g., image data), providing critical information for ADS planning and decision-making. However, advanced ADS, particularly those required to perform the Dynamic Driving Task (DDT) autonomously, are expected to understand driving scenarios across various Operational Design Domains (ODD). This capability requires the support for a continuous comprehension of driving scenarios according to operational data collected by sensors. This paper presents a framework that adopts Graph Neural Networks (GNN) to describe and reason about dynamic driving scenarios via analyzing graph-based data based on collected sensor inputs. We first construct the graph-based data using a meta-path, which defines various interactions among different traffic participants. Next, we propose a design of GNN to support both the classification of the node types of objects and predicting relationships between objects. As results, the performance of the proposed method shows significant improvements compared to the baseline method. Specifically, the accuracy of node classification increases from 0.77 to 0.85, while that of relationships prediction rises from 0.74 to 0.82. To further utilize graph-based data constructed from dynamic driving scenarios, the proposed framework supports reasoning about operational risks by analyzing the observed nodes and relationships in the graph-based data. As a result, the model achieves a MRR of 0.78 in operational risks reasoning. To evaluate the practicality of the proposed framework in real-world systems, we also conduct a real-time performance evaluation by measuring the average process time and the Worst Case Execution Time (WCET). Compared to the baseline models, the results demonstrate the proposed framework presents acceptable real-time performance in analyzing graph-based data.