Optimizing LSTM for sediment load prediction in the Swat river basin, Pakistan: Evaluation of optimizers and activation functions

Accurately estimating sediment load is essential for sustainable water resources management, as excessive sedimentation can reduce reservoir capacity, degrade water quality, and impair aquatic ecosystems, ultimately affecting long-term planning and infrastructure design. However, predictions can be uncertain due to the choice of optimizers and activation functions in machine learning models. This study investigates the influence of seven optimizers (Adam, RMSprop, Adagrad, Adadelta, Adamax, Nadam, and Ftrl) and eight activation functions (ELU, Sigmoid, Linear, Softplus, Swish, SELU, Tanh, and Softmax) on the performance of a Long Short-Term Memory (LSTM) network for sediment load prediction. A total of 56 optimizer-activation combinations were tested using historical data from the Chakdara station in the Swat River basin, Pakistan. The dataset was normalized using Min-Max scaling, with a 70:30 train-test split. Model performance was evaluated using the Coefficient of Determination (R²), Mean Squared Error (MSE), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Percent Bias (PBIAS). Results showed that the Adam-SELU combination achieved the best performance, with an R² of 0.81, MSE of 1363.61, and RMSE of 36.93 during training, and an R² of 0.80, MSE of 2385.2, and RMSE of 48.84 during testing. This configuration also exhibited minimal bias, with PBIAS values of −1.07 (training) and −1.04 (testing). Other combinations, such as Adam-Nadam and Adam-RMSprop, performed relatively well but with higher error metrics. The findings highlight Adam-SELU as the most effective configuration for sediment load prediction, offering improved accuracy. These insights can guide watershed managers in selecting robust models for sediment forecasting.