Tailored vision-language framework for automated hazard identification and report generation in construction sites

Abstract

Timely, comprehensive, and accurate identification of construction hazards is essential for mitigating the accident risk. Automated hazard identification via computer vision has advanced beyond traditional inspection methods but struggles with the dynamic complexity of construction environments, leading to limitations in identifying various hazard categories and generating detailed hazard reports. To address these issues, this study proposes an innovative framework comprising an advanced Vision-Language Model (VLM)-empowered construction hazard identifier, ChatCH, and an end-to-end method for generating construction hazard reports. A dedicated Construction Hazard Dataset (CHD) containing 1,308 real construction hazard images across 32 fine-grained categories was developed for validation purposes. Experimental results show that ChatCH, fine-tuned with the pre-trained VLM Qwen2-VL-7B, achieves a precision of 89.4%, outperforming the pre-trained Qwen2-VL-7B by 43.5% and the traditional pre-trained VLM CLIP by 83.9%. Additionally, ChatCH demonstrates strong few-shot learning capabilities and robustness. Moreover, the end-to-end method for construction hazard report generation can automatically produce structured and detailed hazard reports. This framework provides an innovative solution for construction safety management, enhancing efficiency, accuracy, and automation in construction hazard identification.