Integrating text parsing and object detection for automated monitoring of finishing works in construction projects

Construction process monitoring traditionally relies on manual inspections and document cross-referencing, leading to inefficiencies in project management. Despite advances enabling computer vision-based monitoring and automated document analysis, integrating these technologies remains challenging, particularly in connecting field data with work documentation. This paper proposes an automated monitoring system integrating computer vision-based field data with text-based work instructions. The system employs YOLOv5 object detection models to analyze construction site images and architectural drawings, while utilizing text parsing techniques to extract information from work instructions. Validation using thirty apartment units demonstrated effectiveness in monitoring finishing works, particularly masonry and tiling applications. Results showed consistent performance in establishing automated connections between work instructions, drawings, and site conditions, reducing manual verification requirements while maintaining high accuracy. The successful implementation in finishing works demonstrates potential scalability for broader construction applications with varying complexity levels.