Dual-branch adjacent connection and channel mixing network for video crowd counting

Abstract

This paper focuses on the problem of video crowd counting, which usually uses the spatial and temporal correlations of the consecutive frames to achieve better performance than the single-image crowd counting methods. However, most of the current video crowd counting methods either use only two or three frames for optical flow or frame-difference feature extraction or construct a single-branch network to extract spatiotemporal correlated features. The interactions of features for multiple adjacent frames, which can effectively prevent disturbances caused by background noise, are mostly overlooked. Considering the above problems, we propose a dual-branch adjacent connection and channel mixing network for multi-frame video crowd counting. For the upper branch, an adjacent layer connection method is proposed to capture the multi-scaled spatiotemporal correlations among multiple consecutive frames instead of the traditional dense connections in decomposed 3D convolutional blocks. It achieves better performance and low computation cost. For the lower branch, adaptive temporal channel mixing blocks are proposed to exchange partial channel information among the adjacent frames for feature interaction. The partial channel transpose operation is first proposed to exchange information. It is parameter-free and flexible to achieve interactions among features of any number of consecutive frames. The proposed method outperforms the current image-based and video-based crowd counting models, achieving state-of-the-art performance on six publicly available datasets. The code is available at: https://github.com/aaaabbbbcccccjxzxj/mfvcc.