{"title":"Automated analysis of construction safety accident videos using a large multimodal model and graph retrieval-augmented generation","authors":"Miyoung Uhm , Jaehee Kim , Ghang Lee","doi":"10.1016/j.autcon.2025.106363","DOIUrl":null,"url":null,"abstract":"<div><div>Safety investigators are challenged by the manual task of analyzing large volumes of accident videos through the repetitive process of reviewing them frame by frame, which is both tedious and labor-intensive. This paper proposes the Accident Video Analysis framework (AcciVid), which automates this process using a large multimodal model (LMM) integrated with Graph Retrieval Augmented Generation (Graph RAG). Accident video content and regulations are converted into Resource Description Framework (RDF) triples and stored as graphs, enabling regulation-based analysis through Graph RAG. AcciVid detected 90 %p more potential safety violations than human safety investigators, achieving an F2 Score of 82.4 % compared to their F2 Score of 54.8 %. Furthermore, AcciVid required only an average of 42 s to generate a draft report, whereas human safety investigators needed an average of 4.6 h. This demonstrates AcciVid's potential as an assistant to safety investigators in reducing manual workloads while maintaining high accuracy and efficiency.</div></div>","PeriodicalId":8660,"journal":{"name":"Automation in Construction","volume":"177 ","pages":"Article 106363"},"PeriodicalIF":9.6000,"publicationDate":"2025-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Automation in Construction","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0926580525004030","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"CONSTRUCTION & BUILDING TECHNOLOGY","Score":null,"Total":0}
引用次数: 0
Abstract
Safety investigators are challenged by the manual task of analyzing large volumes of accident videos through the repetitive process of reviewing them frame by frame, which is both tedious and labor-intensive. This paper proposes the Accident Video Analysis framework (AcciVid), which automates this process using a large multimodal model (LMM) integrated with Graph Retrieval Augmented Generation (Graph RAG). Accident video content and regulations are converted into Resource Description Framework (RDF) triples and stored as graphs, enabling regulation-based analysis through Graph RAG. AcciVid detected 90 %p more potential safety violations than human safety investigators, achieving an F2 Score of 82.4 % compared to their F2 Score of 54.8 %. Furthermore, AcciVid required only an average of 42 s to generate a draft report, whereas human safety investigators needed an average of 4.6 h. This demonstrates AcciVid's potential as an assistant to safety investigators in reducing manual workloads while maintaining high accuracy and efficiency.
期刊介绍:
Automation in Construction is an international journal that focuses on publishing original research papers related to the use of Information Technologies in various aspects of the construction industry. The journal covers topics such as design, engineering, construction technologies, and the maintenance and management of constructed facilities.
The scope of Automation in Construction is extensive and covers all stages of the construction life cycle. This includes initial planning and design, construction of the facility, operation and maintenance, as well as the eventual dismantling and recycling of buildings and engineering structures.