Synergizing machine learning and interpolation methods: A Stacking framework for global-scale satellite soil moisture gap filling

Satellite-derived soil moisture (SM) products frequently encounter extensive data gaps that significantly limit their practical utility, necessitating the development of robust gap-filling techniques to generate SM datasets with enhanced accuracy and continuous spatiotemporal coverage. Existing studies have typically relied on single machine learning or interpolation methods to fill SM gaps at regional scales. Machine learning approaches excel at filling missing values in large regions but tend to smooth out important local SM features, while the interpolation methods perform well in areas with low levels of missing data, but exhibit significant uncertainty in regions with large amounts of continuously missing data. These two kinds of approaches show potential complementarity and could together contribute to a more robust gap-filling method, which however have rarely been investigated. To fill this research gap, we established a novel SM gap-filling method by synergizing the advantages of machine learning for large-scale gap filling and the excellent gap-filling performance of interpolation in localized areas using the Stacking method at a global scale. The proposed approach integrates four base models including three machine learning techniques namely Random Forest (RF), Gradient Boosting Decision Tree (GBDT), and Feed-forward Neural Network (FNN), and one interpolation method known as Natural Neighbor Interpolation (NNI), and employs the Least Absolute Shrinkage and Selection Operator (LASSO) as the meta model. We compared the Stacking method and individual approaches in filling ESA CCI missing SM data, and validated the gap-filled SM using extensive ground SM from 1086 sites worldwide. The results indicate: (1) RF performs the best among the six selected machine learning methods, and its overall accuracy at a global scale is higher than that of interpolation methods. The feature importance analysis by SHapley Additive exPlanations (SHAP) indicates ERA5 SM, NDVI, and Global Aridity Index have high importance in the RF gap-filling model; (2) NNI is the best performing approach among the four selected interpolation methods, and it demonstrates better performance than machine learning methods in localized areas where the original SM data is relatively abundant; (3) Stacking is an effective method for SM gap filling on a global scale, with an averaged ubRMSE of 0.017 m³/m³, RMSE of 0.022 m³/m³, Bias of 0.006 m³/m³, and R of 0.87 against the original ESA CCI SM, and it reduces the RMSE by 0.009 m³/m³, ubRMSE by 0.006 m³/m³, and improves R by 0.15 relative to the individual best-performing RF method; (4) The gap-filled SM shows an improved skill than the original ESA CCI SM against global distributed ground SM, with Stacking displaying the lowest ubRMSE of 0.057 m³/m³ and the highest R of 0.63. The proposed Stacking method opens new avenues to fill the gaps in various satellite SM datasets.