Crop and Weeds Classification in Aerial Imagery of Sesame Crop Fields Using a Patch-Based Deep Learning Model-Ensembling Method

Sesame (Sesamum indicum L.) is an important commercial and food crop, and its yields is limited by many insects, pests, diseases, and weeds. Autonomous aerial agrochemicals spray application on sesame fields using a drone aims to save crop from these yield limiting factors and in addition agrochemicals application quantity and site could be controlled, and human health is expected to be protected. For accurate and selective spray application, autonomous systems would need some parameters to distinguish between crop, weed and background. In this research an aerial sesame field dataset has been collected with the focus to classify patch areas of sesame and weeds present in the field. Dataset was captured using Agrocam. We have developed a patch image-based classification approach along with a novel SesameWeedNet convolutional neural network (CNN) inspired by the layer’s configuration of VGG networks and depth-wise convolutions of the MobileNet. The small model contains 6 convolutional layers, and it runs faster and accurately on small patch images. Our approach breaks 1920×1080-pixel images into smaller patch images of size 45×45 pixels. After that, these small patch images are fed to a relatively small CNN for training, validation, and finally for classification. Patch based model ensemble and dataset grouping are two major parts in our methodology. Our system recommends the dataset grouping according to vegetation present in the images to enhance classification results. We have achieved accuracy up to 96.7% with our proposed method. We have tested our system under sunlight variation, in wet and dry soil conditions and at different growth stages. To the best of our knowledge, no attempt has been made to classify and treat crop and weeds in sesame fields at the post-emergence stage previously. In this research we have made the contribution of aerial sesame-weed dataset and a complete deep learning-based approach to classify weeds in sesame fields under variable lighting conditions.