Enhancing Fish Counting in Sonar Images With Multitask Learning and Local–Global Feature Interaction

Abstract

Accurate fish counting is crucial for environmental monitoring and management, with sonar imaging providing a nonintrusive way to gather data in previously inaccessible underwater environments. However, the strong visual similarity between fish and noise in sonar images presents significant challenges in achieving high counting accuracy. Existing methods rely on attention maps to emphasize fish regions but do not fully capture the discriminative features between fish and noise, limiting counting accuracy. To address this, we propose the local-global multitask transformer (LGMFormer) to enhance fish counting in sonar images. The model employs an encoder-decoder architecture, with density map regression as the primary task and multiclass semantic segmentation as an auxiliary task. By predicting multiclass segmentation maps, the shared network layers fully learn discriminative features between fish and noise. We also develop a segmentation-enhanced density head (SEDH) to further strengthen the connection between tasks. Within LGMFormer, the local-global feature interaction (LGFI) module is designed to fuse local spatial detail features with global correlation features for more precise fish counting. Additionally, a high-level feature guidance (HLFG) module is developed to retain more detail during the feature fusion process between the encoder and decoder. We also develop an automatic image segmentation method based on Otsu’s thresholding to create multiclass segmentation labels for sonar images. Extensive experiments on a public sonar fish-counting dataset demonstrate that LGMFormer outperforms state-of-the-art methods in counting accuracy, reducing the mean absolute error (MAE) of count prediction by 17.6% and the false recognition rate by 42.9%. The source code will be available at https://github.com/camerayuhang/LGMFormer