Detecting invasive insects using Uncrewed Aerial Vehicles and Variational AutoEncoders

Abstract

Invasive insect pests, such as the brown marmorated stink bug (BMSB), cause significant economic and environmental damage to agricultural crops. To mitigate damage, entomological research to characterize insect behavior in the affected regions is needed. A component of this research is tracking insect movement with mark-release-recapture (MRR) methods. A common type of MRR requires marking insects with a fluorescent powder, releasing the insects into the wild, and searching for the marked insects using direct observations aided by ultraviolet (UV) flashlights at suspected destination locations. This involves a significant amount of labor and has a low recapture rate. Automating the insect search step can improve recapture rates, reducing the amount of labor required in the process and improving the quality of the data. We propose a new MRR method that uses an uncrewed aerial vehicle (UAV) to collect video data of the area of interest. Our system uses a UV illumination array and a digital camera mounted on the bottom of the UAV to collect nighttime images of previously marked and released insects. We propose a novel unsupervised computer vision method based on a Convolutional Variational Auto Encoder (CVAE) to detect insects in these videos. We then associate insect observations across multiple frames using ByteTrack and project these detections to the ground plane using the UAV’s flight log information. This allows us to accurately count the real-world insects. Our experimental results show that our system can detect BMSBs with an average precision of 0.86 and average recall of 0.87, substantially outperforming the current state of the art.