Real-time automated deep learning based detection and tracking near highway-rail grade crossing for vulnerable road users safety

The vulnerable Road User (VRU) near highway-rail grade crossings (HRGCs) comprises pedestrians, cyclists, and car users. The VRU trespassing violation behavior is the leading cause of highway and railroad related deaths, but many incidents have not been deeply studied. Detection and prevention of such events are critical for road safety improvements, while this task is challenging due to the immense labor costs required for processing streamed video files. This study developed an advanced You Look Only Once (YOLO) deep learning architecture and the Deep Simple Online and Real-time Tracking (Deep SORT) algorithm for real-time VRU trespassing violation detection. Different types of VRUs trespassing were detected near a gated HRGC in Folkston, Georgia. 436 VRU’s trespassing violations were identified in the selected 104-hour video data. The automated VRU’s trespassing detection speed ranged from 43.2 to 654.5 frames per second (FPS), exceeding the field video data recording rate at 30 FPS. The developed methodology resulted in 32 false negatives and 20 false positive detections, with the precision, recall, and F1 values scoring above 92.0%. This work could assist road agencies in reducing VRU’s trespassing violations based on real-time VRU detection and tracking.