PLCFishMOT: multiple fish fry tracking utilizing particle filtering and attention mechanism

Abstract

The task of multi-object tracking of fish fry poses significant challenges, as the majority of the fish fry individuals exhibit highly similar appearances, and the feature distinctions between individual targets are not readily apparent. Consequently, fish tracking algorithms relying primarily on appearance-based features for data association often suffer from low accuracy and poor robustness. To address the challenges inherent in multi-object tracking of fish fry, this study presents an improved DeepSort-based algorithm, dubbed PLCFishMOT, designed specifically for enhanced performance in this domain. Furthermore, the fish fry trajectories may exhibit nonlinear characteristics due to external perturbations. To address this, the original Kalman filtering method has been replaced with a particle filtering approach, which is more suitable for handling nonlinear and non-Gaussian problems. This modification serves to enhance the accuracy of the trajectory prediction process. To further bolster the accuracy of the data association process, the proposed framework incorporates a large separable kernel attention mechanism into the original feature extraction network. This mechanism leverages convolutional kernels of varying sizes to extract target features with differing receptive field dimensions, thereby enhancing the overall effectiveness of the feature representation. The proposed approach effectively addresses the challenge of incorrect ID assignment, which can arise due to the close parallel swimming patterns exhibited by the fish fry. This is achieved by leveraging the cosine angle value between the fry detection frame and the trajectory frame as a discriminating factor. The experimental evaluation of the proposed algorithm on an open-source video dataset demonstrates its strong performance, with the algorithm achieving an IDF1 score of 75.8%, a MOTA score of 98.1%, and IDs is 10, respectively. Furthermore, to assess the generalization capabilities of the proposed approach, validation experiments were conducted using a fish fry video dataset captured in real-world aquaculture scenarios. The experimental results demonstrate that the PLCFishMOT algorithm achieves the best tracking performance compared to other advanced multi-object tracking algorithms.