An Evaluation of RGB-Thermal Image Segmentation for Snowy Road Environment

There has been significant progress in the field of autonomous vehicles in recent years. Many successful attempts to realize self-driving in urban areas have been possible due to the advancement in sensor technology and accelerated computing. However, several challenges exist to achieve autonomous driving in challenging scenarios such as in harsh weather. Inclement weather conditions such as rain, fog, or snow can severely hamper visibility and lead to accidents on the road. Particularly snowy road conditions are challenging due to the slippery road surfaces and hidden lane markings because of snow cover. Such conditions are challenging for autonomous vehicles because of the inability to track distinct visual features in such weather conditions. Most existing image segmentation methods that perform well in clear weather conditions fail in snowy environments. Due to the low gradient of color pixels, the snow-covered objects become a challenge to recognize. This work evaluates some of the state-of-the-art semantic segmentation methods for classifying snow road surfaces using RGB images. We present an entirely new dataset for feature classification in different light conditions (day and night). We tested several existing publicly available deep learning methods and evaluated their efficiency for feature detection in snow conditions. Notably, this work utilizes multiple inputs semantic segmentation techniques to classify snowy road conditions for snow removal machines. Human classification in snow cover is crucial for the safety during the operation of such machines. Therefore we utilize thermal maps and camera images to improve image segmentation efficiency in human detection during snow conditions. The results show that using a thermal map can improve human segmentation efficiency in a snowy environment, especially during the nighttime.