{"title":"Improving 2D displacement accuracy in bridge vibration measurement with color space fusion and super resolution","authors":"Qixuan He, Sen Wang","doi":"10.1016/j.aei.2025.103248","DOIUrl":"10.1016/j.aei.2025.103248","url":null,"abstract":"<div><div>Accurately measuring vibration displacement in bridge structures is crucial for structural health monitoring. Traditional sensor-based methods often limited by high costs and restricted measurement ranges, while vision-based methods provide a simpler, cost-effective alternative for long-distance, non-destructive vibration measurements. Nevertheless, previous visual methods have typically relied on a single color space and overlooked feature preservation during upsampling, resulting in inaccurate detection of small-amplitude targets. This paper proposes a bridge vibration displacement measurement method that integrates multi-color space information with a super-resolution reconstruction module. First, a Color Space Fusion Module converts RGB images to YCbCr and fuses information from both spaces, achieving a balance between performance and parameter count with an increase of less than 0.1M. Next, traditional upsampling is replaced by a learnable super-resolution module to preserve feature information. Finally, a penultimate module integrates global information to improve measurement accuracy before outputting detection results The measurement accuracy and the fit of regressed vibration displacement curves with standard data were validated for the proposed method on beam-like structures in laboratory settings as well as on real large-span bridge structures, with various image acquisition angles and focal lengths. In laboratory settings, the proposed method showed a 6% improvement in mAP over the baseline model YOLOv9c, with a reduction of approximately 54% in average MAE in the time domain across three detection targets, and a 55% reduction in average RMSE. Additionally, on real bridge structure, the proposed method achieved an average MAE of 2.97 and RMSE of 3.84 Compared to existing visual measurement methods, the proposed approach demonstrated superior performance under these diverse conditions.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"65 ","pages":"Article 103248"},"PeriodicalIF":8.0,"publicationDate":"2025-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143627981","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Binyun Wu , Liang Hou , Shaojie Wang , Xiangjian Bu , Cheng Xiang
{"title":"Digital twin modeling for predicting loading resistance of loaders driven by deep transfer learning","authors":"Binyun Wu , Liang Hou , Shaojie Wang , Xiangjian Bu , Cheng Xiang","doi":"10.1016/j.aei.2025.103245","DOIUrl":"10.1016/j.aei.2025.103245","url":null,"abstract":"<div><div>Real-time and accurate prediction of loader loading resistance is the key to achieving autonomous intelligence and energy-saving optimization. This study proposes a deep transfer learning-driven digital twin modeling method to address the challenge of accurately predicting loader loading resistance in scenarios with deficient data from the target conditions. By integrating Convolutional Neural Networks (CNN), Transformer models, and transfer learning techniques, we introduce the CNN-Transformer-TL model. This model effectively transfers knowledge from the source conditions to the target conditions, even when data is deficient, thereby enhancing prediction performance. The model is cross-validated using datasets from sand and loose-soil conditions. The results demonstrate that when sand serves as the source condition and loose-soil as the target domain, prediction accuracy improves by over 10.28%, with computational efficiency increasing by 89.68%. Conversely, when loose-soil is the source condition and sand the target, prediction accuracy improves by more than 20.44%, and computational efficiency rises by 90.45%. Furthermore, compared to other machine learning and deep learning methods, the proposed CNN-Transformer-TL model achieves the highest prediction accuracy and computational efficiency. These findings highlight the model’s superior performance in scenarios with deficient data from the target condition, offering an intelligent and efficient decision-making digital twin model for autonomous loader loading operations.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"65 ","pages":"Article 103245"},"PeriodicalIF":8.0,"publicationDate":"2025-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143627979","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lei Dong , Haojie Zhu , Hanpeng Ren , Ting-Yu Lin , Kuo-Ping Lin
{"title":"Developing YOLOv5s model with enhancement mechanisms for precision parts with irregular shapes","authors":"Lei Dong , Haojie Zhu , Hanpeng Ren , Ting-Yu Lin , Kuo-Ping Lin","doi":"10.1016/j.aei.2025.103257","DOIUrl":"10.1016/j.aei.2025.103257","url":null,"abstract":"<div><div>Given the precision issues caused by part irregularities and occlusion when intelligent assembly technology is used to identify and classify aerospace parts, this study develops a You Only Look Once Version 5 Small (YOLOv5s) model with enhancement mechanisms for detecting precision parts with irregular shapes. First, offsets and learnable parameters are employed in the convolutional layer and combined with the Cross Stage Partial Bottleneck with 3 convolutions (C3)module of the YOLOv5s neck network to enhance the model’s feature extraction capability for irregular objects. Second, the processing speed and recognition accuracy are increased by optimizing the loss function using the smallest possible distance between the corner points of the predicted boxes and the ground truth. Finally, a method is proposed for converting spatial information to channel information in the backbone and neck, thereby reducing information loss and enhancing detection accuracy for small targets. In this study, a dataset of precision parts with irregular shapes based on aerospace components was conducted and experimentally validated. The findings show that the suggested method outperforms YOLOv5 and the most current YOLOv9, increasing identification accuracy to 93.7 % and achieving a speed of 102.04 frames per second. This approach delivers improved detection accuracy for tiny targets, occlusions, and irregularly shaped components as compared to two-stage and one-stage detection algorithms. Furthermore, it maintains a pace of 100 frames per second while striking an optimal balance between speed and accuracy, offering a practical solution for the quick and accurate identification of precision components in smart assembly technology.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"65 ","pages":"Article 103257"},"PeriodicalIF":8.0,"publicationDate":"2025-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143619133","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"CPIR: Multimodal Industrial Anomaly Detection via Latent Bridged Cross-modal Prediction and Intra-modal Reconstruction","authors":"Wen Shangguan , Hongqiang Wu , Yanchang Niu , Haonan Yin , Jiawei Yu , Bokui Chen , Biqing Huang","doi":"10.1016/j.aei.2025.103240","DOIUrl":"10.1016/j.aei.2025.103240","url":null,"abstract":"<div><div>While RGB-based methods have been extensively studied in Industrial Anomaly Detection (IAD), effectively incorporating point cloud data remains challenging. Alongside prevalent memory bank-based approaches, recent research has explored cross-modal feature mapping for multimodal IAD, achieving notable performance and efficient inference. However, cross-modal feature mapping, while effective for detecting anomalies in feature correspondences, struggles to identify those exclusive to a single modality, due to the inherent one-to-many mapping between 2D and 3D data. To overcome this limitation, we propose <strong>Cross-modal Prediction and Intra-modal Reconstruction (CPIR)</strong>, a novel multimodal anomaly detection method. First, we introduce a <strong>Bidirectional Feature Mapping (BFM)</strong> framework that integrates intra-modal reconstruction tasks with cross-modal prediction tasks, enhancing single-modality anomaly detection while maintaining effective cross-modal consistency learning. Second, we propose a novel network architecture, <strong>Latent Bridged Modal Mapping Module (LB3M)</strong>, which introduces a shared latent intermediate state to decouple feature mapping across modalities into mappings between each modality and a shared intermediate state. This design was initially proposed to effectively complete prediction and reconstruction tasks with minimal parameters. However, it also enabled the network to learn more comprehensive feature patterns, significantly improving anomaly detection capabilities. Experiments on the MVTec 3D-AD dataset demonstrate that CPIR outperforms state-of-the-art methods in both anomaly detection and segmentation tasks, while excelling in few-shot learning scenarios.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"65 ","pages":"Article 103240"},"PeriodicalIF":8.0,"publicationDate":"2025-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143619134","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Bufan Liu , Sun Woh Lye , Kai Xiang Yeo , Chun-Hsien Chen
{"title":"A human-centric model for task demand assessment based on unsupervised learning-assisted eye movement measure","authors":"Bufan Liu , Sun Woh Lye , Kai Xiang Yeo , Chun-Hsien Chen","doi":"10.1016/j.aei.2025.103259","DOIUrl":"10.1016/j.aei.2025.103259","url":null,"abstract":"<div><div>Nowadays, Industry 5.0 marks a transformative shift from the focus on efficiency to a human-centered approach, emphasizing the principles of human-AI hybrid systems. This mode prioritizes intelligent technology that supports human abilities rather than replacing them, especially in safety–critical fields like air traffic management (ATM) with human controllers playing an essential role in maintaining safe and efficient operations. Quantifying the task demand of air traffic controllers (ATCOs) is vital to ensure optimal taskload management, thereby assisting in mitigating the risk of human error and promoting sustained operational performance. To realize this aim, this research proposes a human-centric model for task demand assessment based on unsupervised learning-assisted eye movement measure. Initially, two data streams are gathered from human-in-the-loop visual tasks, capturing flight information on the radar screen and eye-tracking data. These data streams are then synchronized and merged by aligning their timestamps. Subsequently, an unsupervised learning-based clustering approach is implemented, utilizing the OPTICS model to identify areas of interest based on aircraft positions, along with the K-Means model to categorize task intensity levels using the derived eye movement data. Finally, a task demand score index is developed for each task intensity level and across all task categories, with parameter weights determined through an entropy-based method. Comprehensive results and analyses are presented to illustrate the method’s applicability and effectiveness. This research paves the way for quantitatively understanding the specific taskload placed on ATCOs.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"65 ","pages":"Article 103259"},"PeriodicalIF":8.0,"publicationDate":"2025-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143627980","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lei Yang , Shaobo Li , Caichao Zhu , Jian Liu , Ansi Zhang
{"title":"Multi-source ensemble transfer learning-based unmanned aerial vehicle flight data anomaly detection with limited data: From simulation to reality","authors":"Lei Yang , Shaobo Li , Caichao Zhu , Jian Liu , Ansi Zhang","doi":"10.1016/j.aei.2025.103255","DOIUrl":"10.1016/j.aei.2025.103255","url":null,"abstract":"<div><div>Flight data anomaly detection is critical for ensuring the safety and reliability of unmanned aerial vehicles (UAVs). Traditional deep learning methods excel when sufficient data is available, but their performance significantly diminishes in data-scarce scenarios. Transfer learning is a promising solution; however, the performance of single-source transfer methods is often limited when there is a significant discrepancy between the source and target domains. This paper proposes a multi-source ensemble transfer learning-based anomaly detection (MSETL-AD) framework, aiming to transfer knowledge from multiple simulated domains to a real domain for anomaly detection in UAV flight data with limited data. First, a similarity calculation method based on dynamic time warping (DTW) is utilized to select simulated source domains that are similar to the target domain to mitigate the negative transfer problem. Second, a modeling strategy based on long short-term memory with attention mechanism (LSTM-AM) integrating transfer learning and fine-tuning techniques is proposed, which constructs a fundamental LSTM-AM prediction model for each source domain and then fine-tunes it using limited data in the target domain during the transfer process. Then, a similarity-based transfer weight assignment method is designed to guide multi-source domains for integration. Next, a similarity-guided dynamic threshold calculation method based on extreme value theory with residual smoothing is introduced to overcome random noise interference and realize adaptive anomaly detection. Finally, the effectiveness of the proposed method is validated through experiments using multiple simulated UAV flight datasets as the source domains and a real UAV flight dataset as the target domain.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"65 ","pages":"Article 103255"},"PeriodicalIF":8.0,"publicationDate":"2025-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143619132","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Infer potential accidents from hazard reports: A causal hierarchical multi label classification approach","authors":"Yipu Qin, Xinbo Ai","doi":"10.1016/j.aei.2025.103237","DOIUrl":"10.1016/j.aei.2025.103237","url":null,"abstract":"<div><div>Inferring categories of accidents caused by hidden hazards detected contributes to safety management and prevention of accidents. For the safety management of many enterprises in a large administrative area, it is necessary to rely on industry experts to review hazard reports produced by front-line employees and infer the categories of potential accidents, which is time-consuming and labor-intensive. In this study, a hierarchical multi-label classification model is proposed to learn a checklist reviewed by industry experts and realize the automatic inference of the accident categories based on hazard descriptions. We simultaneously use the causal effect estimation method designed according to the backdoor adjustment in causal theory to extract the causal part of the text that affects the inference and design a data augmentation method based on the discovered causal knowledge to make the model focus on the causal key information to improve the inference and generalization abilities of the models. From the perspective of theoretical and practical contributions, this study not only realizes the estimation of causal effect of hazard words and the automatic inference of accident categories, which provides support for further accident prevention and safety management. It also makes a successful attempt to apply causality theory combined with deep learning methods in the field of safety, providing a valuable reference for future research on the combination of causal theory and practical applications.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"65 ","pages":"Article 103237"},"PeriodicalIF":8.0,"publicationDate":"2025-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143619713","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Shuo yang , Micaela Demichela , Zhangwei Ling , Jie Geng
{"title":"Evolving process maintenance through human-robot collaboration: An agent-based system performance analysis","authors":"Shuo yang , Micaela Demichela , Zhangwei Ling , Jie Geng","doi":"10.1016/j.aei.2025.103241","DOIUrl":"10.1016/j.aei.2025.103241","url":null,"abstract":"<div><div>Periodic inspections of pressurized vessel systems are essential for maintaining safety through early fault detection. Traditional inspections often expose human operators to hazardous conditions within confined spaces. The advent of inspection robots has shifted the paradigm towards human-robot collaboration (HRC), which seeks to reduce risk while maintaining operational adaptability. This study compared the HRC and fully manual (FM) inspection processes, providing strategic insights for stakeholders. Historically, system performance evaluations have simplified or ignored dynamic human factors. To address this oversight, our research employs Agent-Based Models (ABMs) that encompass the evolving nature of human error, including the impact of fatigue and organizational factors, as well as the variability of human behavior and error recovery mechanisms. Our findings reveal that HRC significantly outperforms FM inspections, enhancing efficiency, accuracy, and safety. Notably, the study confirms that the miss rate of artificial intelligence (AI) for image identification within the HRC process is crucial for reliability and should not fall below the threshold of 0.04. This threshold is a benchmark for AI performance in HRC systems, ensuring that the balance between automated efficiency and human oversight is optimized. The research provides a comprehensive evaluation of HRC in pressurized vessel inspections. It offers a deeper understanding of the complex dynamics involved, advocating for integrating robust AI algorithms to support human operators in safety–critical tasks.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"65 ","pages":"Article 103241"},"PeriodicalIF":8.0,"publicationDate":"2025-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143610255","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An integrated approach for automatic safety inspection in construction: Domain knowledge with multimodal large language model","authors":"Yiheng Wang, Hanbin Luo, Weili Fang","doi":"10.1016/j.aei.2025.103246","DOIUrl":"10.1016/j.aei.2025.103246","url":null,"abstract":"<div><div>This research addresses the challenge of dynamically integrating visual and textual data in construction safety inspections while enhancing adaptability to new safety hazards and ensuring faithful interpretation of safety rules. We propose a novel approach that seamlessly combines multi-modal techniques with domain knowledge, advancing beyond current methods that often struggle with multi-modal understanding and adaptation to new safety hazards. Our approach consists of three key components: (1) a fine-tuned multi-modal LLM for visual and textual processing, (2) a domain knowledge base for evolving safety standards adaptability and output faithfulness, and (3) a multi-step reasoning engine to tackle complex safety inspection tasks. We validate our approach using on-site data from Wuhan subway construction sites, demonstrating its capability to perform moderately accurate (0.57 hazard identification correctness), contextually relevant (0.96 on task relevancy), and faithful safety assessments (0.95 and 0.99 on reasoning faithfulness). The results suggest promising performance in construction scene perception, as well as textual analysis and reasoning. This approach represents an advancement in automatic construction safety inspection and contributes to the broader discourse on formalizing multi-modal processing of construction data, offering insights into creating more flexible and comprehensive safety management systems.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"65 ","pages":"Article 103246"},"PeriodicalIF":8.0,"publicationDate":"2025-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143619131","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jiangzhuo Ren , Rafiq Ahmad , Dejun Li , Yongsheng Ma , Jizhuang Hui
{"title":"Industrial applications of digital twins: A systematic investigation based on bibliometric analysis","authors":"Jiangzhuo Ren , Rafiq Ahmad , Dejun Li , Yongsheng Ma , Jizhuang Hui","doi":"10.1016/j.aei.2025.103264","DOIUrl":"10.1016/j.aei.2025.103264","url":null,"abstract":"<div><div>Digital twins have evolved into a mature concept, unleashing significant potential across diverse domains. The applications of digital twins are currently experiencing a period of rapid growth, with a particular emphasis on the industrial sector. While previous works have examined the frameworks and architectures of digital twins for industrial applications or explored applications within specific industrial fields, there is a gap in the review work concerning the specific characteristics of digital twins in industrial applications and the industrial processes to which this technology has been applied. To address this gap, digital twins’ overall and industrial perspectives are compared through bibliometric analysis to identify the specific development relationship, research hotspots, and knowledge structure of digital twins in industry. Building upon the bibliometric analysis results, this paper presents a complete survey on the technologies/tools supporting industrial applications. The results indicate that simulation, sensor, and cloud computing are predominant in the basic, core, and advanced technologies. Further, this work investigates various industrial processes utilizing digital twins. By combining the bibliometric analysis, it gives that additive manufacturing and machining processes get more attention from digital twins. Finally, according to Shneider’s theory, the evolution stage of digital twins in the industrial context is analyzed. It may have advanced to the late phase of Stage III, a prolific stage.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"65 ","pages":"Article 103264"},"PeriodicalIF":8.0,"publicationDate":"2025-03-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143619135","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}