Indoor scene multi-object tracking based on region search and memory buffer pool

This study proposes a new Indoor Scene Multi-Object Tracking (IS-MOT) task to complete multi-granularity parsing and continuously track indoor human objects. To foster the IS-MOT task, we refer to the basic human movement composition, combining indoor human motion characteristics, constructing a large-scale multi-object tracking benchmark for indoor social robot perspective, termed Multi-Resident Tracking (MRT). To address the issue of insufficient persistent tracking capability when extending existing MOT methods to the IS-MOT task. A persistent visual multi-object tracking method based on region search and memory buffer pool (PeViTrack) is designed. PeViTrack is mainly composed of a Homogeneous Semantic Memory Buffer Pool (HSMBP) that integrates a Motion State Estimation Module (MSEM) and a Hierarchical Matching Correlation Mechanism (HMCM). HSMBP allows the network to construct an allocation representation based on high and low confidence detection boxes, thereby establishing homogeneous and heterogeneous semantic embedding decision spaces in the spatial domain, thus forcing the network to search and accurately associate object homogeneous and heterogeneous features efficiently. Extensive experiments on the constructed MRT and the well-recognized DanceTrack dataset show that PeViTrack achieves state-of-the-art tracking performance. The code and datasets will be made available at https://github.com/funweb/PeViTrack.