DeepSnow/Rain: Light Weather Recognition

Abstract

Weather conditions impact our daily life and transportation. Various sensors, i.e., rain gauges, have been used to monitor weather conditions. However, their implementations are limited, capturing heavier rainfall and snowfall amounts. In contrast, camera image-based sensing is another option, but lighter rainfall and snowfall patterns are hard to be recognized even by state-of-the-art Deep Learning (DL) models despite the indications of heavier events that follow. A single DL is known to deal with limited single tasks for high accuracy. Therefore, this paper proposes DeepSnow/Rain: an integrated DL model consisting of DeepSnow, DeepScene, and DeepRoad. DeepScene is panoptic segmentation of scenes with umbrella and pedestrian recognition. Since it is hard to classify rain or snow with only two objects, road conditions are recognized by implementing DeepRoad. Experimental results in cities show promising results to monitor lighter weather condition changes over time during rainfall or snowfall.