A dual-stream parallel architecture for robust visual tracking using scale-aware region proposals

Abstract

Visual tracking in dynamic environments faces significant challenges such as occlusions, scale variations, and abrupt motion changes, particularly in traffic scenarios. Tracking multi-scale objects and maintaining the temporal correlations across video sequences is essential for accurate tracking. These challenges motivated us to present a novel method that captures long-term dependencies in motion cues using a scale-aware region proposal (SARPro) network that uses a Faster R-CNN pipeline to predict high-quality region proposals for effective multi-scale video object detection and tracking (VODT). The proposed method uses robust feature extraction through a dual-stream feature pyramid network (DS-FPN) that captures spatial and temporal patterns. The SARPro generates precise bounding box proposals, addressing object scale variations. An iterative approach incorporating an LSTM fine-tunes the bounding boxes. A low-confidence track filter (LCTFilter) is integrated into the DeepSORT tracking algorithm to filter out the least confident tracks. The SARPro is designed to operate within multi-threaded parallel computing with GPU acceleration (MTPC-GPU) to optimize simultaneous detection and tracking. Experiments on benchmark datasets demonstrate that SARPro significantly enhances accuracy, achieving robust detection and tracking of small objects in complex video sequences while ensuring real-time performance. SARPro attains mAP scores of 91.35 % with 57.2 FPS on UA-DETRAC and 88.57 % with 41.9 FPS on BDD100K datasets.