A multimodal and meta-learning approach for improved estimation of 3D vegetation structure from satellite imagery

Abstract

This research presents a multimodal and meta-learning approach that integrates multi-source satellite sensor and field plot-level data for enhanced retrieval of 3D vegetation structure. Specifically, the combined effect of integrating multispectral data from Landsat 8 OLI and Sentinel-2 MSI with radar data from Sentinel-1 CSAR was examined. For the utilization of multi-source inputs, the synergistic integration was implemented using efficient machine learning regressors—Random Forest Regressor (RFR) and Extreme Gradient Boosting Regressor (GBR)—ensembled within a meta-learning framework. Three meta-model layers—Multiple Linear Regressor (MLR), K-Nearest Neighbors Regressor (KNR), and RFR—were employed and evaluated. As a subroutine of this integration, a model-specific and data-type-specific feature selection method was employed, which involved training each model on a unique subset of features identified through permutation importance. The idea of the multimodal and meta-learning approach was implemented using extensive plot-wise data from diverse forest types in the New England region utilizing a rich dataset comprising spectral, spectral indices, and backscattering characteristics to capture the variability of forest biomass. The efficacy of multiple ensembling strategies was evaluated, specifically ensembling across data types or regressors, as well as meta-learning across both data types and regressors. Ensembling across data types, which leverages the strengths of both spectral and backscattering information, demonstrated a higher predictive ability, achieving an R2 of 0.68 and an RMSE of 54.21 Mg/ha. This was higher than the ensembling strategy across regressors using the same data type, which yielded an R2 of 0.59 and an RMSE of 61.4 Mg/ha. Nevertheless, the multimodal and meta-learning approach, which collectively leverages both data types and machine learning regressors, achieved superior performance, with an R2 of 0.82 and an RMSE of 40.5 Mg/ha. This was significantly greater than a conventional ensemble method, which lacked the meta-layer integration. Additionally, the meta-model layer using RFR yielded better results compared to the KNR or MLR layers, demonstrating the capability of RFR in handling complex interactions across variables. These results highlight the superior accuracy and reliability of the multimodal and meta-learning approach, indicating its substantial potential to enhance precision in ecological monitoring and carbon management.