An enhanced IWCARS method for measuring soil available potassium

Abstract

The Competitive Adaptive Re-weighted Sampling (CARS) method, while excelling in feature extraction, encounters several challenges when processing low-quality data, including high computational complexity, intricate parameter settings, and the potential for overfitting. To address these issues, this paper introduces the IWCARS (Initial Weight and Weight, I & W) algorithm, which implements two key methodological enhancements: initial weight selection and weight update strategy. This algorithm, building upon the traditional CARS algorithm and density-based clustering, offers a supplementary tool for data feature selection by computing density and weight, and employs an adaptive model evaluation mechanism to select the most pertinent features, ultimately constructing a model with enhanced predictive capability. IWCARS optimizes model performance by dynamically adjusting the feature set, thereby improving the algorithm's prediction performance and model fit. Furthermore, the IWCARS method, in conjunction with a Partial Least Squares (PLS) model, was applied to measure soil Available Potassium (AK) content using near-infrared spectroscopy. Five pre-processing techniques were conducted on the near-infrared spectrum, with the IWCARS + PLS model constructed using first derivative data, yielding optimal results. The experimental results demonstrated that the model based on 1st Derivative + IWCARS + PLS yielded the best performance. Specifically, the model achieved R_C² of 0.9905, R_p² of 0.9817, RMSEC of 0.8917, RMSEP of 0.9024, and RPD of 8.5176. Robustness, versatility, and transferability tests demonstrated that the proposed IWCARS algorithm, when integrated into the PLS model, achieved commendable measurement accuracy. While there are limited strategies for concurrently addressing high computational complexity, challenging parameter settings, and overfitting risks, this study aims to mitigate these concerns by reducing the computational complexity of the CARS algorithm, simplifying parameter settings, and preventing overfitting, ultimately enhancing the model's fitting accuracy, training speed, and generalization capability.