Multi-label material and human risk factors recognition model for construction site safety management

Introduction: Construction sites are prone to numerous safety risk factors, but safety managers have difficulty managing these risk factors for practical reasons. Moreover, manually identifying multiple risk factors visually is challenging. Therefore, this study aims to propose a deep learning model–based multi-label risk factor recognition (MRFR) framework that automatically recognizes multiple potential material and human risk factors at construction sites. The research answers the following questions: How can a deep learning model be developed and optimized to recognize and classify multiple material and human risk factors automatically and concurrently at construction sites, and how can the decision-making process of the model be understood and improved for practical application in preemptive safety management? Methods: Data comprising 14,605 instances of eight types of material and human risk factors were collected from construction sites. Multiple risk factors can occur concurrently; thus, an optimal model for multi-label recognition of possible risk factors was developed. Results: The MRFR framework combines material and human risk factors into a single label while achieving satisfactory performance with an F1 score of 0.9981 and a Hamming loss of 0.0008. The causes of mispredictions by MRFR were analyzed by interpreting the decision basis of the model using visualization. Conclusion: This study found that the model must have sufficient capacity to detect multiple risk factors. Performance degradation in MRFR is primarily due to difficulties recognizing visual ambiguities and a tendency to focus on nearby objects when perspective is involved. Practical applications: This study contributes to safety management knowledge by developing a model to recognize multi-label material and human risk factors. Furthermore, the results can be used as guidelines for data collection methods and model improvement in the future. The MRFR framework can be used as an algorithm to recognize risk factors preemptively and automatically at real-world construction sites.