Meta-styled CNNs: boosting robustness through adaptive learning and style transfer

Abstract

Recent studies reveal that standard Convolutional Neural Networks (CNNs)—conventionally struggle—when the training data is corrupted, leading to significant performance drops with noisy inputs. Therefore, real-world data, influenced by various sources of noise like sensor inaccuracies, weather fluctuations, lighting variations, and obstructions, exacerbates this challenge substantially. To address this limitation—employing style transfer on the training data has been proposed by various studies. However, the precise impact of different style transfer parameter settings on the resulting model’s robustness remains unexplored. Therefore, in this study, we systematically investigated various magnitudes of style transfer applied to the training data, assessing their effectiveness in enhancing model robustness. Our findings indicate that the most substantial improvement in robustness occurs when applying style transfer with maximum magnitude to the training data. Furthermore, we examined the significance of the dataset’s composition from which the styles are derived. Our results demonstrate that utilizing a limited subset of just 64 diverse, randomly selected styles is adequate to observe desired performance generalization even under corrupted testing conditions. Therefore, instead of uniformly selecting styles from the dataset, we developed a probability distribution for selection. Notably, styles with higher selection probabilities exhibit qualitatively distinct characteristics compared to those with lower probabilities, suggesting a discernible impact on the model’s robustness. Utilizing style transfer with styles having maximum likelihood according to the learned distribution led to a 1.4% increase in mean performance under corruption compared to using an equivalent number of randomly chosen styles.