Wh-AI-les: Exploring harmonized vision models robustness against distribution shift

Abstract

The remarkable and increasing efficiency of learning-based vision strategies has induced strong paradigm shift in favor of neural architectures that are consequently finding their way into real-world applications with significant impact. Nevertheless, neural networks display a particular brittleness that can significantly hurt performance when deployed outside lab conditions, which is symptomatic of the independent and identically distributed (i.i.d.) hypothesis violation. This lack of consistency across domains is hurtful since it lessens reliability and trustworthiness and could have severe consequences in sensitive environments such as automated driving. Building on recent progresses in vision strategy harmonization and, through a flexible data generation process, this work explores the impact of alignment over robustness for a wide set of models as well as an encouraging theoretical configuration that uses aligned saliency features through a brute-force approach.