A two-stage 3D multi-fish tracking model using patch-based underwater stereo matching

Abstract

Indoor recirculating aquaculture systems necessitate the extensive application of automatic monitoring technologies due to the inherent complexity and precision involved. Underwater three-dimensional (3D) tracking is essential for monitoring fish. However, underwater imaging in aquaculture environments frequently encounters challenges such as image blurring, low contrast, and excessive noise. These problems reduce the effectiveness of stereo matching algorithms, subsequently affecting the precision of binocular-based underwater 3D tracking. As a result, current applications often rely on expensive hardware, such as Time-of-Flight (TOF) cameras or structured light cameras. To achieve high-precision underwater 3D tracking economically with binocular cameras, a two-stage 3D multi-fish tracking (TMT) model was proposed. The TMT model concentrates on processing fish-containing patches, eliminating complex and extensive background regions in underwater images. This improves contour and spatial position accuracy in stereo matching, reduces computational demands, and simplifies point cloud filtering. Specifically, in stage 1, a detector and tracker identify fish-containing patches within stereo images. Stage 2 uses a novel patch-based underwater stereo matching method to match patches of the same fish in the left and right images, then calculates and filters the fish point cloud. Evaluations of positioning accuracy showed that the Mean Absolute Error (MAE) between model results and the true values is less than 2 cm in both horizontal and vertical directions, and less than 5 cm in the depth direction. Furthermore, the TMT model demonstrates stable performance during a 7-h hypoxia experiment on fish, effectively identifying various fish behavioural changes.