A deep generative approach for crash frequency model with heterogeneous imbalanced data

Crash frequency model is often subject to excessive zero observation because of the rare nature of crashes. To address the problem of imbalanced crash data, a deep generative approach – augmented variational autoencoder – was proposed to generate synthetic crash data for the association measure between crash and possible explanatory factors. This approach was characterized by a factorized generative model and refined objective function. For instance, the generative model can handle heterogeneous data including real-valued, nominal and ordinal distributions. On the other hand, the refined objective function can control for the random effect by better recognizing both the zero-crash and non-zero crash cases. In this study, comprehensive traffic and crash data of multiple distribution types in Hong Kong in the period between 2014 and 2016 were used. To assess the data generation performance of the proposed augmented variational autoencoder method, a conventional data synthesis technique (synthetic minority oversampling technique-nominal continuous) was also considered. Performances of crash frequency models of total crashes and fatal and severe injury crashes are assessed. For total crashes, the results of parameter estimation, in terms of statistical fit, prediction accuracy, and explanatory factors identified, of the crash frequency model based on synthetic data using the augmented variational autoencoder method adhered closer to that based on original data, compared to that based on synthetic data using the synthetic minority oversampling technique-nominal continuous method. For fatal and severe injury crashes, zero-crash observations were prevalent, with the ratio of zero-crash to non-zero crash cases of 9 to 1. Crash data was first balanced using the proposed augmented variational autoencoder method. Then, fatal and severe injury crash frequency models using correlated random parameter models based on original data and balanced data were estimated respectively. Results indicate that fatal and severe injury crash frequency model based on balanced data outperforms its counterpart, with the lowest root mean square error, lowest mean absolute error, and highest number of crash explanatory factors identified. More importantly, correlation between the random parameters can be revealed. Findings of this study should shed light to both researchers and practitioners for the development of crash frequency models, with which the problem of excessive zero observations is prevalent when highly disaggregated traffic and crash data by time and space are used.