Residential energy consumption dynamics: A SHAP-based interpretation, k-means clustering, and predictive modelling

Abstract

A proper understanding of climatic consumption dynamics is critical for demand-side management, grid stability, and climate-sensitive residential energy planning. The non-linear interaction between meteorological conditions has necessitated intelligent predictive models. However, existing studies focused on machine learning (ML) models in their black-box nature, with limited interpretability of the impact of climatic-drivers, climatic-demand interactions, and hidden consumption regimes. This research fills this gap through an integrated framework that combines seasonal hypothesis-testing, k-means clustering, Shapley Additive exPlanations (SHAP)-based interpretability, and advanced predictive modeling using XGBoost, Random Forest (RF), long-short term memory (LSTM), Support Vector Machine (SVM), and Autoregressive Integrated moving average (ARIMA). The seasonal hypothesis-testing using ANOVA and Tukey's HSD revealed statistically significant differences in energy consumption across seasons, with peak-demand during winter and summer extremes. SHAP-based feature ranking identified temperature and humidity as the most influential drivers of electricity-demand. The k-means clustering revealed three distinct groups/clusters, which reflect the climatic-consumption scenarios. The ensemble learning (RF and XGBoost) exhibited the lowest training-error. RF had the best training performance with MSE, RMSE, and MAE values of 1.6484, 1.2839, and 0.8480. The data-driven insights in this study provide useful intelligence that supports critical decision-making through a proper understanding of the residential climatic-consumption dynamics.