An explainable hybrid transformer–BiLSTM framework for multivariate household energy demand forecasting with weather integration

Abstract

Purpose

Accurate and explainable household load forecasting is critical for demand-side management, tariff-aware scheduling, and reliable smart grid operation. This study introduces a leakage-controlled multi-horizon forecasting pipeline that integrates predictive accuracy with statistical validation, interpretability, robustness, and operational relevance.

Methods

We model multivariate household demand using an hourly smart-meter dataset spanning 14 months (Nov 2022-Jan 2024; N=10,234 time steps), incorporating aligned local weather covariates. A hybrid Transformer-BiLSTM is trained using a multi-output configuration to predict 24-hour and 168-hour trajectories. Hyperparameters are optimized via Bayesian optimization (Optuna) employing chronological train/validation/test splits and a rolling-origin evaluation protocol. Performance is assessed using MAE, RMSE, and MAPE, while pairwise forecast differences are validated using the Diebold–Mariano procedure. Model explanations are generated through SHAP and attention analyses, further complemented by robustness testing (noise and feature dropout) and inference-efficiency measurements.

Results

At the 24-hour horizon, the hybrid model achieves MAE and RMSE values of 0.0539/0.0701 (MAPE 29.7%), yielding performance comparable to N-BEATS (MAE/RMSE 0.0531/0.0684). For the 168-hour horizon, the model attains superior performance among evaluated baselines (MAE/RMSE/MAPE 0.0566/0.0746/30.1%) and demonstrates statistically significant improvements over the Transformer, BiLSTM, and TCN models (Diebold-Mariano, \(p<0.001\)). SHAP analysis identifies electrical indicators (e.g., voltage, power factor) and meteorological variables (e.g., pressure, temperature statistics) as the dominant drivers of medium-term predictions. Median inference latency remains in the tens-of-milliseconds range per sample, facilitating near real-time application.

Conclusion

Beyond improving forecast accuracy, the proposed framework provides statistically supported and interpretable attributions, remains stable under input degradation, and demonstrates operational value in a tariff-based battery scheduling case study, reducing energy cost by approximately 2.29% over an aggregated multi-week evaluation.

Graphical abstract