Quantification of Data Imbalance

Abstract

In this article, we propose a novel approach to quantify the imbalance in data, addressing a significant gap in the field of regression analysis. Real-world datasets often exhibit an inherent imbalance in their data distribution, which adversely affects learning algorithms such as those used in neural networks. This results in less accurate learning of rare occurrences and a model bias towards more frequent cases, posing challenges in scenarios where rare events are crucial, like energy load prediction. While many solutions exist for classification problems with imbalanced data, regression problems lack adequate research. To address this, we introduce a method to quantify data imbalance by defining it as the disparity between the probability distribution of the data and a relevance-associated distribution. Our approach includes various metrics that can handle multivariate data, allowing for the identification of imbalanced samples and the abstract quantification of imbalance through the mean imbalance ratio. This method facilitates the comparison of regression datasets based on their imbalance, providing insights into dataset quality and evaluating data resampling techniques. We validate our approach using synthetic data and compare it to established metrics such as the Kullback–Leibler divergence and the Wasserstein metric. Furthermore, analysis of real datasets shows a moderate correlation between sample rarity and the approximation error of neural networks, extreme gradient boosting trees and random forests, indicating that underrepresented samples are linked to higher approximation errors.