How Does Oversampling Affect the Performance of Classification Algorithms?

Abstract

To address the issue of imbalanced datasets classification, this study explores how different oversampling algorithms and imbalance ratios affect the performance of classification algorithms. Two oversampling algorithms, including random oversampling and Synthetic Minority Oversampling Technique (SMOTE), are used to adjust the imbalance ratio of the training dataset to 999:1, 99:1, 9:1, 3:1, and 1:1. Four classification methods, including the Convolutional Neural Network, Vision Transformer, XGBoost and CatBoost, are evaluated using performance metrics such as precision, recall, AUC, and F2-Score. We conduct more than 240 experiments and observe that oversampling ratio has a significant positive impact on AUC and recall rate, but a negative impact on precision. The study also identifies the best oversampling algorithm and imbalance ratio for each classification algorithm. It is noteworthy that the Vision Transformer algorithm used in this study has not been employed in previous research on imbalanced data classification.