Multimodal data fusion for ergonomic assessment of construction workers in visually obstructed environments

Abstract

Work-related musculoskeletal disorders are the leading cause of nonfatal injuries in the construction industry. Ergonomic assessment methods can effectively prevent these disorders. Vision-based ergonomic risk assessment methods are widely applied in construction sites due to their cost-effectiveness and non-invasiveness. However, existing vision-based methods often face challenges in accurately estimating worker pose in real construction sites with visually obstructed environments, such as self-obstruction, object obstruction, and body parts out of view. Additionally, these methods lack consideration of external load factors for ergonomics. To overcome these issues, this paper proposes a multimodal ergonomic assessment method, combining visual data and pressure signals. Multimodal method integrates pressure and visual data in a unified feature space, improving pose estimation results and providing external load metrics for a more comprehensive ergonomic assessment. Field experiments show that the accuracy of pose estimation and risk assessment is enhanced, supporting the safety and health of construction workers.