Rotation translation matrices analysis method for action recognition of construction equipment

Abstract

Lack of intelligence is one of the primary factors hindering the implementation of video surveillance. Action recognition is a method used to enhance the effectiveness of video surveillance and has garnered the interest of numerous researchers. In recent years, the advancement of deep learning (DL) frameworks has led to the proposal of numerous DL-based action recognition models. However, most of these models exhibit poor performance in recognizing actions of construction equipment, primarily due to the presence of multiple targets in complex real-life scenes. Considering the above information, we have developed a method for action recognition that involves analyzing the motion of the fundamental components of construction vehicles. Firstly, we estimate the essential components of construction vehicles from the video inputs using an instance segmentation method. Secondly, to assess the motion state of the robotic arm of the equipment, we have developed an analysis method based on rotation and translation (RT) matrices. We propose to examine the relationship between action recognition of construction vehicles and RT matrices. The evaluations of the respective datasets were conducted. The experimental results validate the effectiveness of the proposed framework, and our model demonstrates state-of-the-art performance in action recognition of construction equipment. We utilize RT matrices to model the degrees of movement of construction equipment, allowing us to analyze their actions and providing a unique perspective on action recognition. We believe that the proposed framework can facilitate the transition of video surveillance techniques from research to practical applications, ultimately generating economic value.