Comparison of Waymo rider-only crash data to human benchmarks at 7.1 million miles

Abstract

Objectives

This article examines the safety performance of the Waymo Driver, an SAE level 4 automated driving system (ADS) used in a rider-only (RO) ride-hailing application without a human driver, either in the vehicle or remotely.

Methods

ADS crash data were derived from NHTSA’s Standing General Order (SGO) reporting over 7.14 million RO miles through the end of October 2023 in Phoenix, Arizona, San Francisco, California, and Los Angeles, California, and compared to human benchmarks from the literature.

Results

When considering all locations together, the any injury reported crashed vehicle rate was 0.6 incidents per million miles (IPMM) for the ADS vs. 2.80 IPMM for the human benchmark, an 80% reduction or a human crash rate that is 5 times higher than the ADS rate. Police-reported crashed vehicle rates for all locations together were 2.1 IPMM for the ADS vs. 4.68 IPMM for the human benchmark, a 55% reduction or a human crash rate that was 2.2 times higher than the ADS rate. Police-reported crashed vehicle rate reductions for the ADS were statistically significant when compared in San Francisco and Phoenix, as well as combined across all locations and the any-injury-reported reductions were statistically significant in San Francisco and in all locations. The any property damage or injury comparison had statistically significant decreases in 3 comparisons but also nonsignificant results in 3 other benchmarks. When excluding ADS crashes with a delta-V less than 1 mph (a measure of sensitivity to lower reporting threshold), about half of the ADS collisions were excluded, resulting in comparisons that showed a large statistically significant reduction in all comparisons except for one comparison from San Francisco.

Conclusions

The statistically significant reductions in police-reported and any injury reported crash rates indicate a promising positive safety impact of ADS. The direction and significance of comparisons in the any property damage or injury outcome group are inconclusive due to difficulties in estimating a matching human benchmark. More research is needed on defining any property damage or injury benchmarks with clear lower reporting thresholds to reduce the systematic uncertainty in the benchmark rates. Together, these crash rate results contribute to the continuous growth in confidence, together with other methodologies, in a safety case approach.