Deep Learning-Based Instance Segmentation of Mushrooms in Their Natural Habitats

Fungi can be used as the environmental bioindicators of a given area. Detection and localization of mushrooms in their natural habitats represent an important task that can help scientists and conservationists to classify them and carefully study their interaction with the microclimate. Mushrooms are difficult to identify due to the significant variability of their macroscopic features. To address this, the current work aims to provide the accurate and efficient way of identifying various mushroom species in their natural environments. In this article, a comprehensive dataset of annotated mushroom images was created to test the detection performance of five deep instance segmentation architectures (i.e., mask region-based convolutional neural network (Mask R-CNN), Mask Scoring R-CNN, Cascade Mask R-CNN, Hybrid Task Cascade, and DetectoRS). In addition, the study also compares various convolutional neural network (CNN)-based and visual transformer-based backbone feature extraction components for Mask R-CNN using a set of evaluation metrics. The results showed that the proposed instance segmentation models, which employed transfer learning and fine-tuning, adequately identified mushroom instances despite the complex backgrounds. The Mask R-CNN model architecture with ResNeXt as a backbone was superior to visual transformers. Overall, DetectoRS was the best model to detect mushrooms in various complex natural habitats and reached satisfactory results for instance segmentation (mean average precision = 0.69; recall = 0.79; and F 1-score = 0.74), producing well-defined individual mushroom masks. The findings of this study will support the development of a digital tool for the automated detection and segmentation of various mushroom instances in a wide range of natural environments.