Simultaneous estimation of multiple soil properties from vis-NIR spectra using a multi-gate mixture-of-experts with data augmentation

Abstract

Simultaneous estimation of multiple soil properties from vis-NIR hyperspectra presents a cost-effective and time-efficient approach. Previous studies have utilized multi-task convolutional neural network (multi-CNN) with share-bottom structures based on the hard parameter sharing. However, multi-CNN often ignores the differential characteristics of correlations between soil properties, limiting the accuracy of soil property estimation. The multi-gate mixture-of-experts network (MMoE) offers a solution by extracting both common features across all soil properties and unique features specific to each soil property, which probably could provide better estimation outcomes than the conventional shared-bottom multi-CNN. In the present study, a MMoE was built based on a total of 17,272 mineral soil samples from the Land Use/Cover Area Frame Survey (LUCAS) topsoil database that includes vis-NIR spectra with ten physicochemical properties, i.e., clay, silt, sand, pH (in water), organic content (OC), calcium carbonate (CaCO₃), nitrogen (N), phosphorous (P), potassium (K), and cation exchange capacity (CEC). To evaluate the performance of MMoE, a series of other models were also built, i.e., partial least square regression (PLSR), single-task convolutional neural network (single-CNN), multi-task convolutional neural network (multi-CNN) and multi-task long short-term memory (multi-LSTM). Furthermore, performance of feature-spectrum selected by competitive adaptive reweighted sampling (CARS) on the accuracy of the MMoE was also explored, as well as a data augmentation method of stacking raw spectra with five preprocessed spectra data. The results demonstrated that MMoE had higher accuracy than PLSR, single-CNN, and multi-LSTM models, with RMSE reduction of 5 %–48 %, R² improvement of 1 %–119 %, and CCC improvement of 0 %–74 %. Compared with multi-CNN, MMoE showed better accuracy for all properties except pH, with RMSE reduction of 3 %–8 %, R² improvement of 1 %–12 %, and CCC improvement of 0 %–5 %. However, the feature-spectrum selected by CARS did not improve the accuracy of MMoE compared to full-band spectrum, whereas the data augmentation method was effective in improving the estimation accuracy of MMoE compared to raw spectra, with RMSE reduction of 14 %–28 %, R² improvement of 3 %–88 %, and CCC improvement of 1 %–63 %. Consequently, this study proves that MMoE based on data augmentation is an efficient and accurate method for the simultaneous estimation of multiple soil properties from vis-NIR spectra.