RainSD: Rain style diversification module for image synthesis enhancement using feature-level style distribution

Abstract

Autonomous driving technology nowadays targets to level 4 or beyond, but the researchers are faced with some limitations for developing reliable driving algorithms in diverse challenges. To promote the spread of autonomous vehicles widely, it is important to address safety issues in this technology. Among various safety concerns, the sensor blockage problem by severe weather conditions can be one of the most frequent threats for multi-task learning-based perception algorithms during autonomous driving. To handle this problem, the importance of generating proper datasets is becoming more significant. In this paper, a synthetic road dataset with sensor blockage generated from real road dataset BDD100K is suggested in the format of BDD100K annotation. Rain streaks for each frame were made using an experimentally established equation and translated utilizing the image-to-image translation network based on style transfer. Using this dataset, the degradation of the diverse multitask networks for autonomous driving, such as lane detection, driving area segmentation, and traffic object detection, has been thoroughly evaluated and analyzed. The tendency of performance degradation of deep neural network-based perception systems for autonomous vehicles has been analyzed in depth. Finally, we discuss the limitation and future directions of deep neural network-based perception algorithms and autonomous driving dataset generation based on image-to-image translation.