Cross-scale content adaptive network for three-dimensional multi-object tracking and fish activity quantification

Tracking and quantifying fish activity are vital for evaluating their health status and adaptability to the environment. However, most current research on fish tracking and activity quantification suffers from the limitation of being two-dimensional, losing crucial vertical or horizontal information. To facilitate tracking and quantitative analysis of fish activity in three-dimensional (3D) space, a cross-scale content-adaptive network-based 3D multi-object tracking method for fish is proposed, through which fish movements are quantified accordingly. Firstly, a cross-scale content-adaptive fusion network is proposed to accurately determine the fish positions from top-down and side views, thereby mitigating the issue of scale variation across different perspectives. Secondly, a hierarchical tracking method is implemented to obtain the 3D trajectories of the fish, addressing the challenge of cross-view identity matching. Finally, activity parameters in 3D space, including the activity quantity and trajectory length for individual fish, as well as the dispersion and cohesion for the fish group, are calculated. The proposed method was validated, achieving a Multi-Object Tracking Accuracy (MOTA) of 97.68% and an Identification F1 Score (IDF1) of 97.93%. For activity quantification, the Mean Absolute Error (MAE) was found to be 0.088 (unit weight·(cm/s)²), and the Root Mean Square Error (RMSE) was 0.1064 (unit weight·(cm/s)²). These results affirm the method’s adaption of fish features across scales for 3D tracking and activity analysis. With its efficient performance, our method presents as an instrument for activities such as fish behavior monitoring, selective breeding, and environmental assessment.