Deep learning-based automated method for enhancing excavator activity recognition in far-field construction site surveillance videos

Abstract

Vision-based classifiers, highly sensitive to camera placement, face significant challenges under far-field conditions at construction sites. To address these challenges, this paper proposes a deep learning-based method for enhancing excavator activity recognition using a 3D Residual Neural Network (3D ResNet) classifier with transfer learning. Machine learning-based SHapley Additive exPlanations (SHAP) analysis was employed to evaluate classifier performance across varying camera placements, focusing on distance, height, and angle. Additionally, an image preprocessing method for object enlargement and clarity enhancement was introduced to improve accuracy. Key findings include: (i) optimal weighted F1-score of 0.866 achieved with camera placement at 20 m distance, 6 m height, and 45° angle; (ii) SHAP analysis identifying distance as the most critical factor; (iii) weighted F1-score of 0.818 obtained with real-world far-field video after applying the proposed image preprocessing. The proposed method demonstrates potential for enhancing productivity and carbon emissions management through precise excavator activity monitoring.