Development of an action classification method for construction sites combining pose assessment and object proximity evaluation

Addressing the inherent hazards of on-site construction work and stagnant labor productivity is crucial in the construction industry. To tackle these challenges, automated monitoring of construction sites and analysis of workers' actions play a pivotal role. In this study, we developed a method for classifying actions at a construction site from video, using deep learning. Specifically, we used two image processing techniques, pose assessment and object detection, and found that the accuracy of action classification was improved by extracting information on the proximity of workers to equipment installed at the construction site, and also by considering the pose information. For classification, LSTM (Long Short-Term Memory), CNN (Convolutional Neural Network), and XGBoost models were used, and the presence of proximity information improved average recall by 7.0% to 8.5% for all models used. The final model was developed as an ensemble of these methods, offering accuracy and average recall that are higher than with conventional methods. The methodology developed in this research enables quantification and visualization of work content at construction sites, contributing to the overall enhancement of safety and productivity within the construction industry.