Improving the performance of machine learning algorithms for detection of individual pests and beneficial insects using feature selection techniques

To reduce damage caused by insect pests, farmers use insecticides to protect produce from crop pests. This practice leads to high synthetic chemical usage because a large portion of the applied insecticide does not reach its intended target; instead, it may affect non-target organisms and pollute the environment. One approach to mitigating this is through the selective application of insecticides to only those crop plants (or patches of plants) where the insect pests are located, avoiding non-targets and beneficials. The first step to achieve this is the identification of insects on plants and discrimination between pests and beneficial non-targets. However, detecting small-sized individual insects is challenging using image-based machine learning techniques, especially in natural field settings. This paper proposes a method based on explainable artificial intelligence feature selection and machine learning to detect pests and beneficial insects in field crops. An insect-plant dataset reflecting real field conditions was created. It comprises two pest insects—the Colorado potato beetle (CPB, Leptinotarsa decemlineata) and green peach aphid (Myzus persicae)—and the beneficial seven-spot ladybird (Coccinella septempunctata). The specialist herbivore CPB was imaged only on potato plants (Solanum tuberosum) while green peach aphids and seven-spot ladybirds were imaged on three crops: potato, faba bean (Vicia faba), and sugar beet (Beta vulgaris subsp. vulgaris). This increased dataset diversity, broadening the potential application of the developed method for discriminating between pests and beneficial insects in several crops. The insects were imaged in both laboratory and outdoor settings. Using the GrabCut algorithm, regions of interest in the image were identified before shape, texture, and colour features were extracted from the segmented regions. The concept of explainable artificial intelligence was adopted by incorporating permutation feature importance ranking and Shapley Additive explanations values to identify the feature set that optimized a model's performance while reducing computational complexity. The proposed explainable artificial intelligence feature selection method was compared to conventional feature selection techniques, including mutual information, chi-square coefficient, maximal information coefficient, Fisher separation criterion and variance thresholding. Results showed improved accuracy (92.62 % Random forest, 90.16 % Support vector machine, 83.61 % K-nearest neighbours, and 81.97 % Naïve Bayes) and a reduction in the number of model parameters and memory usage (7.22 × 10⁷ Random forest, 6.23 × 10³ Support vector machine, 3.64 × 10⁴ K-nearest neighbours and 1.88 × 10² Naïve Bayes) compared to using all features. Prediction and training times were also reduced by approximately half compared to conventional feature selection techniques. This demonstrates a simple machine learning algorithm combined with an ideal feature selection methodology can achieve robust performance comparable to other methods. With feature selection, model performance can be maximized and hardware requirements reduced, which are essential for real-world applications with resource constraints. This research offers a reliable approach towards automatic detection and discrimination of pest and beneficial insects which will facilitate the development of alternative pest control approaches and other targeted pest removal methods that are less harmful to the environment than the broad-scale application of synthetic insecticides.