Live Tracking of Rail-Based Fish Catching on Wild Sea Surface

Abstract

Automated video analysis in fishery has drawn increasing attention since it is more scalable and deployable in conducting survey, such as fish catch tracking and size measurement, than traditional human observers. However, there are challenges from the wild sea environment, such as the rapid motion of the tide and the white water foam on the surface, which can create large noise in video data. In this work, we present an innovative method for live tracking of rail-based fish catching by combining background subtraction and motion trajectories techniques in highly noisy sea surface environment. First, the foreground masks, which consist of both fish and tide-blob noise, are obtained using background subtraction. Then, the fish are tracked and separated from noise based on their trajectories, and their boundaries are further refined with histogram of optical flow. Finally, the segmentation is acquired with a dense conditional random field (CRF) in which the optical flow on trajectories are transformed and served as feature vectors for calculating the pairwise potential. Our experimental results demonstrate that the trajectories and the feature vectors from optical flow greatly improve the tracking performance.