Crowd semantic segmentation based on spatial-temporal dynamics

Crowd semantic segmentation is supposed to not only accurately segment the crowd into groups but also describe them by semantic properties. We define a group as a set of members sharing common spatial-temporal dynamics, i.e., motion consistency and distribution homogeneity. This paper proposes a novel crowd semantic segmentation method, termed as joint spatial-temporal semantic segmentation, which leverages the temporal motion characteristics and spatial distribution information of crowd. We first conduct temporal motion grouping and spatial distribution grouping according to motion consistency and distribution homogeneity respectively. Then, a a joint semantic segmentation algorithm is employed to combine the motion and distribution groups into semantic groups. States of these groups are described in terms of motion pattern and density level. Experiments show that our proposed method is effective to obtain favorable segmentation with semantic descriptions.