NICE k metrics: Unified and multidimensional framework for evaluating deterministic solar forecasting accuracy

Abstract

Accurate solar energy output prediction is key to grid stability and efficient energy management. However, conventional error metrics (such as Root Mean Squared Error ($RMSE$), Mean Absolute Error ($MAE$), coefficient of determination ($R^2$) and Skill Scores ($SS$)) fail to capture the multidimensional complexity of solar irradiance forecasting. They lack forecastability sensitivity, depend on arbitrary baselines (e.g., clear-sky models) or adapt poorly to operational needs. To address these limitations, this study introduces $NIC{E}^k$ (Normalized Informed Comparison of Errors, $k=1,2,3,\Sigma$), a robust, flexible, and multidimensional evaluation framework. Each score is tied to an $L^k$ norm: $NIC{E}^1$ targets average errors, $NIC{E}^2$ emphasizes large deviations, $NIC{E}^3$ amplifies outliers, and $NIC{E}^{\Sigma }$ aggregates all contributions. Validation relied on synthetic Monte Carlo trials and real data from Spain SIAR network (68 stations across diverse climates). Benchmark models include autoregressive methods, Extreme Learning, and smart persistence, covering both linear and machine learning strategies. Theoretical $NICE$ metrics matched empirical values only under strict assumptions ($R^2\sim 1.0$ for $NIC{E}^2$). In contrast, $NIC{E}^{\Sigma }$ consistently outperformed conventional metrics in discriminative power ($p<0.05$ vs $p>0.05$ for $nRMSE$/$nMAE$). Over longer horizons, $NIC{E}^{\Sigma }$ maintained significant p-values ($10^{-6}$ to 0.004), unlike $nRMSE$ and $nMAE$. In addition, conventional metrics (including $nMBE$ and $R^2$) failed to distinguish models in pairwise tests, whereas $NIC{E}^k$ achieved $p<0.001$ with wider, normally distributed values, enhancing comparability. Theoretical and empirical findings confirm the framework’s sensitivity and operational relevance. These results support adopting $NIC{E}^k$ as a unified, interpretable, and robust alternative to conventional metrics for deterministic solar forecasting.