Deep learning techniques for Video Instance Segmentation: A survey

Video Instance Segmentation (VIS), also known as multi-object tracking and segmentation, represents a fundamental challenge in computer vision that requires simultaneous detection, segmentation, and tracking of object instances across video frames. This complex task has gained significant attention due to its crucial role in various real-world applications. The advent of deep learning has promoted VIS approaches, leading to numerous architectural innovations and performance improvements. This survey presents a systematic review of deep learning-based VIS methods, introducing a novel categorization based on temporal modeling strategies: frame-by-frame, clip-based, in-memory feature propagation, and in-memory object query propagation. Comprehensive quantitative comparisons of existing work across three major VIS benchmark datasets are also provided. Additionally, emerging challenges in the field are explored, with several promising research directions identified, aiming to provide valuable insights for researchers and practitioners interested in VIS, while further advancing deep learning techniques for VIS.