On the robustness of action recognition methods in compressed and pixel domain

This paper investigates the robustness of two state-of-the-art action recognition algorithms: a pixel domain approach based on 3D convolutional neural networks (C3D) and a compressed domain approach requiring only partial decoding of the video, based on feature description using motion vectors and Fisher vector encoding (MV-FV). We study the robustness of the two algorithms against: (i) quality variations, (ii) changes in video encoding scheme, (iii) changes in resolutions. Experiments are performed on the HMDB51 dataset. Our main findings are that C3D is robust to variations of these parameters while the MV-FV is very sensitive. Hence, we consider C3D as a baseline method for our analysis. We also analyze the reasons behind these different behaviors and discuss their practical implications.