Detecting Potholes from Dashboard Camera Images Using Ensemble of Classification Mechanisms

Abstract

Road damage such as potholes may occur on roads due to aging, which may affect traffic. Periodic inspections of road damages are difficult due to the high cost of road surveys. The development of a system that automatically detects potholes and other road damages from dash cam images can allow inexpensive road inspections and can overall improve the problem of the long-term overlook of road damages. Last year, we conducted a demonstration experiment in Edogawa City, Tokyo, using an existing image-based road damage detection method. From that experiment, we found that the detection of potholes on actual roads often causes false positives in detecting shadows and manholes. In this study, we propose a method to reduce false positives in pothole detection, which was considered to be a problem through the demonstration experiment. Since the evaluation based on a pothole-only dataset is not practical, we constructed a dataset for evaluation by adding shadow and manhole images. Our method consists of two main components: data augmentation and an ensemble of classification mechanisms for object detection models. The result of the test on the reconstructed pothole dataset showed that the Average Precision (AP), which is a measure to evaluate the performance of object detection, and F1, which is the harmonic mean of precision and recall, were improved compared to the existing method. Our new method is expected to be an effective pipeline for tasks and situations where false positives are likely to occur and where false positives are more considered as an issue than false negatives, given that they are not dependent on the domain of potholes.