EL-NRF: Enhancing ensemble learning for regression with a noise reduction framework

Abstract

Ensemble learning aims to improve predictive accuracy by combining multiple models, with stacking being a widely adopted technique that employs a meta-learning framework. Despite significant advancements in stacking-based ensemble models, improving their robustness and generalization remains a persistent challenge. In this study, a two-phase noise reduction approach is proposed to improve the performance of stacking ensembles in regression tasks. In the first phase, feature-space noise is reduced through dimensionality reduction using Truncated Singular Value Decomposition (TSVD), which eliminates redundant and less informative components. In the second phase, sample-level noise is mitigated by applying a statistical thresholding method to identify and exclude high-residual instances. The proposed approach is evaluated on a real-world delivery time prediction dataset and six public benchmark datasets. Experimental results demonstrate that the integration of noise reduction techniques significantly enhances the predictive performance of stacking models, with improvements ranging from 1.65 % to 23.81 %, even in scenarios where conventional stacking fails to outperform its base learners. These results highlight the importance of noise reduction in improving the generalization capability of ensemble models, particularly in real-world regression problems.