The Goldilocks Principle: Achieving Just Right Boundary Fidelity for Long-Tailed Classification

Abstract

This study addresses the challenges of learning from long-tailed class imbalances in deep neural networks, particularly for image recognition. Long-tailed class imbalances occur when a dataset's class distribution is highly skewed, with a few head classes containing many instances and numerous tail classes having fewer instances. This imbalance becomes problematic when traditional classification methods, especially deep learning models, prioritize accuracy in the more frequent classes, neglecting the less common ones. Furthermore, these methods struggle to maintain consistent boundary fidelity—decision boundaries that are sharp enough to distinguish classes yet smooth enough to generalize well. Hard boundaries, often caused by overfitting tail classes, amplify intra-class variations, while overly soft boundaries blur distinctions between classes, reducing classification accuracy. We propose a dual-branch network with a shared feature extractor to overcome these challenges. This network uses instance and median samplers for head and medium classes and a reverse sampler for tail classes. Additionally, we implement a specialized loss function as a feature regularizer to reduce the model's sensitivity to irrelevant intra-class variations during classification. This loss function dynamically modulates feature representation alignment, promoting cohesive intra-class structures and clear inter-class separations. To achieve this, our framework incorporates two key components: Dual-Branch Sampler-Guided Mixup (DBSGM) and Adaptive Class-Aware Feature Regularizer (ACFR), which work together to balance class representation and refine decision boundaries. Integrating DBSGM and ACFR during training helps shape decision boundaries that align with class semantics. To ensure class boundaries are appropriately defined, we propose the temperature-adaptive supervised contrastive loss (TASCL) within the ACFR module, achieving the right balance between smoothness and sharpness. Our single-stage, end-to-end framework demonstrates significant performance improvements, offering a promising solution to the challenges of long-tailed class imbalances in deep learning.