FoCA: Failure-oriented Class Augmentation for Robust Image Classification

Image classification with classes of varying difficulty can cause performance disparity in deep learning models and reduce the overall performance and reliability of the predictions. In this paper, we introduce a failure-oriented class augmentation (FoCA) technique to address the problem of imbalanced performance in image classification, where the trained model has performance deficits in some of the dataset's classes. By employing Generative Adversarial Networks (GANs) to augment these deficit classes, we finetune the model towards a balanced performance among the different classes and an overall better performance on the whole dataset. Unlike earlier works, during training, our method focuses on those classes with the lowest accuracy after the initial training phase. Only these classes are augmented to boost the accuracy, which leads to better performance. FoCA is designed to be used with a light-weight GAN method to make the GAN-based augmentation viable and effective, even for datasets with only few images per class, while simultaneously requiring less computation than other, more complex GAN methods. Our implementation of FoCA combines this light-weight GAN method for class-wise data augmentation with state-of-the-art deep neural network techniques for training. Experiments show an overall improvement from FoCA with competitive or better accuracy than the previous state-of-the-art on five datasets with different sizes and image resolutions.