A hybrid modelling framework of machine learning and extreme value theory for crash risk estimation using traffic conflicts

Extreme value theory is the state-of-the-art modelling technique for estimating crash risk from traffic conflicts, with two different sampling techniques, i.e. block maxima and peak-over-threshold, at its core. However, the uncertainty associated with the estimates obtained by these sampling techniques has been too large to enable its widespread practical use. A fundamental reason for this issue is the improper selection of extreme values and a lack of a suitable and efficient sampling mechanism. This study proposes a hybrid modelling framework of machine learning and extreme value theory to estimate crash risk from traffic conflicts with an efficient sampling technique for identifying extremes. More specifically, a machine learning approach replaces the conventional sampling techniques with anomaly detection techniques since an anomaly is a data point that does not conform with the rest of the data, making it very similar to the definition of an extreme value. Six representative machine learning-based unsupervised anomaly detection algorithms have been tested in this study. They include iforest, minimum covariance determinant, one-class support vector machine, k-nearest neighbours, local outlier factor, and connectivity-based outlier factor. The extremes identified by these algorithms are then fitted to extreme value distributions for both univariate and bivariate frameworks. These algorithms were tested on a large set of traffic conflict data collected for four weekdays (6 am to 6 pm) from three four-legged intersections in Brisbane, Australia. Results indicate that the proposed hybrid models consistently outperform the conventional extreme value models, which use block maxima and peak-over-threshold as the underlying sampling technique. Among the sampling algorithms, iforest has been found to perform better than other algorithms in estimating crash risks from traffic conflicts. The proposed hybrid modelling framework represents a methodological advancement in traffic conflict-based crash estimation models and opens new avenues for exploring the possibility of utilising machine learning techniques within the existing traffic conflict techniques.