How to detect occluded crosswalks in overview images? Comparing three methods in a heavily occluded area

Abstract

Crosswalk presence data are crucial for pedestrian safety and urban planning. However, obtaining such data at a large scale is often challenging due to the high cost associated with traditional collection methods. While automated methods based on computer vision have been explored to detect crosswalks from aerial images, a major obstacle to their application is the handling of candidate crosswalks occluded by objects or shadows in the aerial imagery. To address this challenge, this study explores different deep learning-based solutions, including the aerial-view method (AVM) and street-view method (SVM), which are commonly used, and a combination of them, i.e., the dual-perspective method (DPM). Deep learning models based on convolutional neural networks (CNNs) with the VGG16 architecture were trained using 16 815 images to automatically detect crosswalks from both aerial and street view images. To compare the performance of these methods in handling occlusions, 1 378 images from a heavily occluded area were processed separately by the three methods. The results showed that the AVM suffered the most when dealing with images from a heavily occluded area, resulting in the lowest accuracy, precision, recall, and F1 score among the three methods. On the other hand, the SVM outperformed the AVM significantly. The DPM demonstrated the highest accuracy and precision values, indicating its superiority in accurately predicting the location of a crosswalk. However, the SVM exhibited the highest recall value, highlighting its superior ability to recover an occluded crosswalk among all methods.