Performance Evaluation of Activation Functions in Deep Residual Networks for Short-Term Load Forecasting

Abstract

Short-Term Load Forecasting (STLF) is essential for ensuring efficient and reliable power system operations, requiring accurate predictions of electricity demand. Deep Residual Networks (DRNs), with their ability to mitigate gradient vanishing and model complex nonlinear relationships in load data, have emerged as a powerful tool for STLF. This study evaluates the performance of various activation functions within DRN models, focusing on their impact on predictive precision and generalization. Experiments were conducted using the DRN architecture for STLF on two distinct datasets: ISO-NE and Malaysia. The findings demonstrate that activation functions significantly influence the predictive performance of DRN-based STLF models. Specifically, the DRN model using Swish achieved the best results on the ISO-NE dataset (Mean Absolute Percentage Error, MAPE = 1.3806%), while the DRN model with Hyperbolic Tangent (Tanh) excelled on the Malaysia dataset (MAPE = 4.9809%). These results underscore the importance of aligning activation function selection with dataset characteristics to optimize the performance of DRN models in STLF. This study provides valuable insights for advancing STLF research and guiding practical applications in load forecasting.