A Study on Missing Data Imputation Methods for Improving Hourly Solar Dataset

Abstract

In the era of big data, large period of missing data is a common problem which affect the data quality and final forecasting results if not handled properly. Therefore, filling missing data in datasets is importance since the most of real-time datasets have a huge number of missing values. This paper first gives a comprehensive overview of various imputation methods for filling missing data. Then proposes a technique based on a popular Multivariate Imputation by Chained Equation (MICE) to fill numeric data in PV dataset. Finally analyses the impact of this technique and compares the performance with other imputation algorithms. For practice, this study uses historical measurement PV generation from the North PV site of Taiwan, and Numerical Weather Prediction (NWP) data consists of solar irradiance, temperature, sea level pressure, humidity, rainfall, wind speed. The NWP dataset is provided by Taiwan Central Weather Bureau (CWB) which is called Deterministic Weather Research and Forecasting (WRFD). Experimental results showed that the proposed imputation algorithm can improve short-term PV generation forecasting accuracy based on RMSE.