MdeBEIA: Multi-task deep leaning for butterfly ecological image analysis

Abstract

Butterfly ecological image analysis (BEIA) is an exciting and essential field where computer vision can significantly aid in ecological research and biodiversity conservation. Although deep learning has made significant strides in BEIA, the existing models still handle tasks such as segmentation and classification independently, which constrains the potential performance improvements gained by exploiting the correlations between these tasks. In this paper, we design a multi-task deep learning model, named MdeBEIA, to perform both segmentation and classification tasks for BEIA. The MdeBEIA model features a unified encoder that extracts global semantics and spatial information from butterfly images, creating a shared feature representation for both tasks. This approach leverages the intrinsic correlations between segmentation and classification to enhance feature learning. To further boost classification performance, we integrate a Region of Interest Guidance Module (RIGM), which uses intermediate segmentation masks and a self-attention mechanism to refine feature learning by emphasizing contextual relationships. Additionally, we employ a deep mutual learning strategy to improve the model's performance and generalization ability. Experimental results show that MdeBEIA achieves a Jaccard score of 94.70 % in segmentation, surpassing the state-of-the-art by 0.93 %, with comparable inference speeds. In classification, it outperforms the state-of-the-art by 0.81 %, reaching 98.34 %.