Deconstructing deep imbalanced regression: a comprehensive review and experimental evaluation

Abstract

In real-world applications, there is a fundamental problem: the data most critical to predict interesting events, anomalies, and high-stakes outliers are the rarest, while less interesting data is abundant. Although deep learning is deployed specifically for these difficult prediction tasks, data-driven models inevitably fail in underrepresented areas. This discrepancy between the empirical data- and the desired evaluation distribution is equivalent to a target distribution shift. The research field, termed Deep Imbalanced Regression (DIR), has emerged explicitly to address this challenge, which is particularly acute for continuous targets where most conventional classification-based methods are ill-suited. In this paper, we present the first comprehensive review of the DIR landscape, organized around a novel two-axis taxonomy that disentangles challenges along a Data Axis (target distribution shift, continuity, and density) and a Deep-Learning Axis (shared capacity, biased updates, and manifold distortion), where the latter captures a cascading failure mechanism through which deep models systematically neglect underrepresented targets. Within this framework, we systematically categorize and analyze 19 state-of-the-art methods spanning architectural, algorithm-level, and representation learning approaches, and empirically re-evaluate twelve of them with publicly available implementations under controlled, identical conditions. To stress-test generalization across the full target range, we introduce three novel targeted evaluation protocols, Balanced Extrapolation, Bimodal Interpolation, and Blind-Spot Isolation, that expose failure modes hidden by standard benchmarks (https://github.com/noah-puetz/deconstructing_deep_imbalanced_regression). Our study underscores the significant impact of imbalance on regression accuracy, offering a conceptual framework and practical benchmarks to catalyze further development of systems capable of capturing the rare as reliably as the common.