Key frame extraction for falling detection

Abstract

The continuous advancement of computer vision and deep learning technology provides powerful technical support for supervising the occurrence of falling behaviors. Considering that some frames in the video contribute little to the recognition of falling behaviors, in order to eliminate the video frames irrelevant to falling behaviors, this paper proposes an algorithm to extract key frames in a video by combining LUV local maximum and Mask-RCNN. Firstly, candidate key frames are extracted based on the local maximum of LUV. Video frames containing motion changes can be obtained. Afterwards, in order to further eliminate the video frames that are less relevant to the fall action, Mask-RCNN is used for human body detection. According to the aspect ratio of the bounding box and the motion speed of the human body, the video frames that are more relevant to the fall action are selected as key frames. Experiments and results analysis are carried out on the UR fall detection dataset, Multiple cameras fall dataset, Le2i Fall detection dataset and falling video dataset of real scenes. The effectiveness and accuracy of the proposed method are verified.