High-quality rainy image generation method for autonomous driving based on few-shot learning

Rainy image generation aims to transfer images from standard domains such as daytime into rainy domains. Related researches can be divided into unsupervised methods and supervised methods according to use of semantic label constraints. The generalization ability of unsupervised methods is highly related to the domain gap between rain-free images and rain-effect images, which is difficult to keep the layout consistency due to the lack of semantic constraints on the paired data of the target domain. In supervised generative models, the scarcity of paired datasets has a serious impact on the performance of generative results. Moreover, most of the existing rainy paired datasets are synthesized by simply merging and the rain simulated by noise, which can be very different from the images shot in natural condition. So, in order to improve the realism of rainy image generation, we proposed a realistic paired rainy dataset (PRD) in autonomous driving scenes to explore the real rain representations and fusion mechanism with clear images. Besides, aiming at lack of paired samples in autonomous driving scenarios, we are committed to the study of the few-shot learning in generative models. An incremental hybrid training strategy is proposed to make full use of a few datasets. Through extensive experiments, we verify the effectiveness of our proposed method, which achieves more realistic results on limited labeled data. In the future, the dataset can be applied in many other tasks of autonomous driving.