Adapting a global plant identification model to detect invasive alien plant species in high-resolution road side images

Abstract

Early detection of invasive alien plant species is crucial for addressing their environmental impact. Recent advancements in vehicle-mounted equipment enable automatic analysis of high-resolution images to detect invasive plants along roadsides, a primary vector for their spread. Deep learning technologies show promise for processing this data efficiently, but the choice of approach significantly affects both computational and human resource costs. Object detection and segmentation methods require costly annotations, making them impractical for scaling to the thousands of invasive species worldwide. In contrast, multi-label classification, i.e. to predict all species present in the image, is less demanding but still challenging to implement without many annotated images for numerous species. However, large datasets from citizen science platforms such as Pl@ntNet or iNaturalist offer rich visual data for classifying individual plant species. In this article, we assess whether large plant identification models trained on such data can be leveraged for species detection in high-resolution images. Specifically, we explore two approaches: a multi-label classification model and a tiling-based model, using a vision transformer from the Pl@ntNet platform. We evaluate these models on high-resolution roadside images, both using a pre-trained model without fine-tuning and after applying fine-tuning. Our findings indicate that the tiling approach significantly outperforms other methods without fine-tuning and shows a slight advantage when fine-tuning is applied, demonstrating significant potential for detecting thousands of species without task-specific adaptation.