Closed-Loop Tracking-by-Detection for ROV-Based Multiple Fish Tracking

Abstract

Fish abundance estimation with the aid of visual analysis has drawn increasing attention based on the underwater videos from a remotely-operated vehicle (ROV). We build a novel fish tracking and counting system followed by tracking-by-detection framework. Since fish may keep entering or leaving the field of view (FOV), an offline trained deformable part model (DPM) fish detector is adopted to detect live fish from video data. Besides that, a multiple kernel tracking approach is used to associate the same object across consecutive frames for fish counting purpose. However, due to the diversity of fish poses, the deformation of fish body shape and the color similarity between fish and background, the detection performance greatly decreases, resulting in a large error in tracking and counting. To deal with such issue, we propose a closed-loop mechanism between tracking and detection. First, we arrange detection results into tracklets and extract motion features from arranged tracklets. A Bayesian classifier is then applied to remove unreliable detections. Finally, the tracking results are modified based on the reliable detections. This proposed strategy effectively addresses the false detection problem and largely decreases the tracking error. Favorable performance is achieved by our proposed closed-loop between tracking and detection on the real-world ROV videos.