S3H: Long-tailed classification via spatial constraint sampling, scalable network, and hybrid task

Abstract

Long-tailed classification is a significant yet challenging vision task that aims to making the clearest decision boundaries via integrating semantic consistency and texture characteristics. Unlike prior methods, we design spatial constraint sampling and scalable network to bolster the extraction of well-balanced features during training process. Simultaneously, we propose hybrid task to optimize models, which integrates single-model classification and cross-model contrastive learning complementarity to capture comprehensive features. Concretely, the sampling strategy meticulously furnishes the model with spatial constraint samples, encouraging the model to integrate high-level semantic and low-level texture representative features. The scalable network and hybrid task enable the features learned by the model to be dynamically adjusted and consistent with the true data distribution. Such manners effectively dismantle the constraints associated with multi-stage optimization, thereby ushering in innovative possibilities for the end-to-end training of long-tailed classification tasks. Extensive experiments demonstrate that our method achieves state-of-the-art performance on CIFAR10-LT, CIFAR100-LT, ImageNet-LT, and iNaturalist 2018 datasets. The codes and model weights will be available at https://github.com/WilyZhao8/S3H