Advanced Engineering Informatics最新文献

{"title":"Graph Neural Networks for building and civil infrastructure operation and maintenance enhancement","authors":"Sajith Wettewa,&nbsp;Lei Hou,&nbsp;Guomin Zhang","doi":"10.1016/j.aei.2024.102868","DOIUrl":"10.1016/j.aei.2024.102868","url":null,"abstract":"<div><div>This systematic review, conducted within the PRISMA framework, investigates the disruptive capabilities of Graph Neural Networks (GNNs) in optimising Operations and Maintenance (OM) practices within the building and civil infrastructure domain. Addressing 5 research questions and encompassing 111 studies from 2014 to 2024, our study identifies the multifaceted applications of GNNs across different project stages from data enhancement to operational scenario enhancement. When considering integrated Facilities Management (FM) approaches, GNNs are employed for data enhancement purposes, leveraging techniques such as semantic enrichment of Building Information Modelling (BIM), various data imputation scenarios, and semantic segmentation of point clouds to enhance data quality and completeness. Operational scenarios involve the utilisation of GNN algorithms for anomaly detection, fault classification, system optimisation, and forecasting. Methodological optimisations crucial for GNN feasibility include feature engineering, architecture optimisation to balance complexity and overfitting risk, and the integration of Explainable Artificial Intelligence (XAI) methods to enhance model validity and trust. Physical principles integration through Physics-Informed Graph Neural Networks (PIGNNs) further enhances model explainability and validation. Future research directions focus on data interoperability enhancement, scalability improvements, and explainability enhancements. Automated graph generation and labelling, heterogeneous GNN models, supporting algorithms such as Long Short-Term Memory (LSTM) and reinforcement learning are proposed to overcome analysis limitations. Specific workflows targeting building performance-based semantic enrichment, building systems data imputation, and interdependency prediction are proposed in future directions. The review highlights the symbiotic relationship between GNN-based analysis and digital twin data analysis, emphasising the suitability of GNNs in addressing the demands of digital twin data analysis in the building and civil infrastructure domain.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102868"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142532179","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A conflict clique mitigation method for large-scale satellite mission planning based on heterogeneous graph learning","authors":"Xiaoen Feng,&nbsp;Minqiang Xu,&nbsp;Yuqing Li","doi":"10.1016/j.aei.2024.102915","DOIUrl":"10.1016/j.aei.2024.102915","url":null,"abstract":"<div><div>For the large-scale and intensive demands of satellite remote sensing observation, the complexity of constraint relationships grows explosively with expansion of satellite task scale. How to efficiently deal with the complex and temporal varying constraint conflicts, and mine the implicit knowledge existing among satellite mission constraints, which is significant to enhance scheduling efficiency, however, is also a core difficulty in the satellite scheduling problem. In this paper, we propose a conflict clique mitigation method based on dynamic task-constrained heterogeneous graph learning to solve large-scale satellite mission scheduling. The method exploits the advantage of heterogeneous graphs to characterize multiple unstructured relationships, and projects the temporal-varying features of constraint conflicts to the spatial topology of multiple nodes and edges in a heterogeneous graph. Thus, a dynamic constraints heterogeneous graph model for satellite tasks based on sampling critical conflict cliques is developed. And an improved heterogeneous attention network with quadratic unconstrained binary optimization (HAN-QUBO) is proposed, which is able to deal with the heterogeneous graphs and attempts to represent the implicit principles of multiple constraints of satellite missions, so that the valuable strategies and experiences of conflict mitigation can be extracted. The simulation experiments demonstrate that the method can provide effective empirical guidance for multi-satellite scheduling, greatly relieve the pressure of cumbersome constraint conflict checking process for large-scale tasks. The average number of conflict resolutions has been reduced by about 73.48 % for EOSSPs with tens of thousands tasks, while the quality of solutions is maintained at the same time, which significantly improves the efficiency of multi-satellite scheduling.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102915"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142578042","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Dynamic airport gate assignment with improved Shuffled Frog-Leaping Algorithm and triangle membership function","authors":"Hsien-Pin Hsu ,&nbsp;Wan-Fang Yang ,&nbsp;Tran Thi Bich Chau Vo","doi":"10.1016/j.aei.2024.102888","DOIUrl":"10.1016/j.aei.2024.102888","url":null,"abstract":"<div><div>The rapid development of the air transportation industry has increased air traffic, posing challenges to the task of airport gate assignment (AGA) for flights. Most past studies have solved the AGA problem (AGAP) using deterministic models, which are incapable of dealing with uncertainty and dynamic conditions at airports. Thus, this research employs fuzzy theory and proposes a triangular membership function to handle flight uncertainty in the AGAP. In addition, an improved metaheuristic, termed the improved Shuffled Frog-Leaping Algorithm (ISFLA), is proposed to circumvent the computationally intractable problems commonly faced by exact approaches when handling large instances. In this research, the AGAP is first formulated as a stochastic Mixed-Integer Linear Programming (MILP) model, with stochastic flight lateness and earliness considered. The objective of this model is to minimize the total cost, which consists of three sub-costs: passenger walking distances, non-preferred gate (NPG) assignments for planes, and fuzzy idle times of gates. These three sub-costs correspond to the major concerns of passengers, airlines, and airports, respectively. The cooperation between the ISFLA and the triangular membership function demonstrates their capability to effectively handle big AGAP instances. Furthermore, the experimental results show that the ISFLA outperforms the standard SFLA, Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and Firefly Algorithm (FA).</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102888"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142586520","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"FedITA: A cloud–edge collaboration framework for domain generalization-based federated fault diagnosis of machine-level industrial motors","authors":"Yiming He,&nbsp;Weiming Shen","doi":"10.1016/j.aei.2024.102853","DOIUrl":"10.1016/j.aei.2024.102853","url":null,"abstract":"<div><div>Adequate samples are necessary for establishing a high-performance supervised learning model for intelligent fault diagnosis. Startup companies may only have normal devices and therefore there exists extreme class imbalance of training samples. Lack of faulty devices makes it difficult to independently establish supervised learning. The ideal aggregated training using raw data from multiple client sources may lead to potential conflicts of interest, making it difficult to implement. In addition, individual difference caused by manufacturing inconsistencies and dynamic testing environments is a special interference for machine-level industrial motors, which is more significant in the information flow of multiple client sources. This article proposes a federated iterative learning algorithm (FedITA) as a cloud–edge collaboration framework for domain generalization-based federated fault diagnosis of machine-level industrial motors. The proposed FedITA utilizes progressive training and iterative weight updates to enhance secure interaction between different clients, effectively reducing the risk of overfitting caused by extreme class imbalance. A hybrid perception mechanism is implemented by developing complementary perception modules and integrated into a hybrid perception field network (HPFNet) as a recommended global federated model. The proposed method and model are performed on real production line signals and can achieve mean cross-machine F1-score of 96.50% in limited communication.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102853"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142417122","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A novel semi-supervised prediction modeling method based on deep learning for flotation process with large drift of working conditions","authors":"Fanlei Lu,&nbsp;Weihua Gui,&nbsp;Liyang Qin,&nbsp;Xiaoli Wang,&nbsp;Jiayi Zhou","doi":"10.1016/j.aei.2024.102934","DOIUrl":"10.1016/j.aei.2024.102934","url":null,"abstract":"<div><div>Deep neural networks have been broadly utilized for soft sensing modeling for the process performance which is significant for process control but cannot be measured online. However, the popular deep learning models still cannot adapt to large drift of working conditions in the process industry, which causes the model accuracy to become worse and worse with the time go on. Moreover, the cost of acquiring sufficient labeled data is very high. Therefore, in this study, a semi-supervised deep learning method called dynamic multi-scale selective kernel network (DMS-Sknet) with novel loss function is proposed by taking the flotation process as the case. In DMS-SKnet, multiscale features are extracted from froth images by using multi-scale dilated convolution kernel, and then fused with other process data in time series. A channel attention module with soft attention is designed to learn the important relationships between multi-scale feature maps and process features. Finally, based on the semi-supervised Mean-teacher (MT) learning framework, a new loss function is proposed, in which temporal distance is considered to improve the generalization ability and the long-term accuracy of the network. The experimental results using industrial flotation process data show that this method can effectively improve the grade prediction accuracy after a long period of significant changes in the working conditions.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102934"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142659118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A task-oriented theil index-based meta-learning network with gradient calibration strategy for rotating machinery fault diagnosis with limited samples","authors":"Mingzhe Mu,&nbsp;Hongkai Jiang,&nbsp;Xin Wang,&nbsp;Yutong Dong","doi":"10.1016/j.aei.2024.102870","DOIUrl":"10.1016/j.aei.2024.102870","url":null,"abstract":"<div><div>In industrial scenarios, rotating machinery operates in harsh environments under complex and variable conditions, which leads to a scarcity of available data. This brings challenges to intelligent model-based rotating machinery fault diagnosis. For this issue, a task-oriented theil index-based meta-learning network with gradient calibration strategy (TTIMN-GCS) is proposed for rotating machinery fault diagnosis with limited samples. Firstly, a fine-grained feature learner (FGFL) is designed to extract high-dimensional fine-grained fault information from limited samples. The FGFL is modeled after the human recognition process of fine-grained objects, enhancing distinguishing between fault categories with subtle differences. Secondly, a task inequality metric named task-oriented theil index is developed to acquire more competitive update rules from limited samples, which creatively frees the initial performance of the meta-FGFL from being overly tied to specific tasks. Finally, a gradient calibration strategy is proposed to adjust the optimization trajectory of TTIMN-GCS, which facilitates the diagnostic model evolution toward robust generalization performance. Four diagnostic cases on several datasets are designed, and the diagnostic accuracies under the 5-shot setting reach 98.18 %, 96.68 %, 94.60 %, and 93.90 %, respectively, which are better than other state-of-the-art methods. Experimental results exhibit that the TTIMN-GCS has a remarkable capability to identify new fault categories from a few samples and is potentially promising for engineering applications.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102870"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142442569","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Hybrid physics-embedded recurrent neural networks for fault diagnosis under time-varying conditions based on multivariate proprioceptive signals","authors":"Rourou Li,&nbsp;Tangbin Xia,&nbsp;Feng Luo,&nbsp;Yimin Jiang,&nbsp;Zhen Chen,&nbsp;Lifeng Xi","doi":"10.1016/j.aei.2024.102851","DOIUrl":"10.1016/j.aei.2024.102851","url":null,"abstract":"<div><div>Accurate fault diagnosis for industrial robots is imperative to improve their availability. Proprioceptive signals collected by intrinsic sensors of robot joint servo drive systems provide a nonintrusive and promising way for practical in-situ diagnosis. However, they generally exhibit significant non-stationarity owing to time-varying operation conditions and limited sampling frequencies constrained by system hardware, which poses challenges in fault signature identification. Thus, a hybrid physics-embedded recurrent neural network is proposed for robot fault diagnosis under variable operation conditions based on proprioceptive signals. It embeds robot governing ordinary differential equations (ODE) as an inductive bias to account for known dynamics. Concurrently, tailored neural networks (NN) are leveraged to compensate for unmodeled dynamics residuum and unmeasurable health states, efficiently extending the hypothesis space. Hereinto, system status-represented latent space inferred from observations is comprehensively regularized by state reconstruction, fault classification, and Fisher discrimination losses to promote state representability and class distinguishability. Furthermore, a bilinear layer-based NN is constructed to statistically model intrinsic nonlinearities simplified away by physical models. Finally, the model-based and data-driven components are synergistically integrated by a differentiable ODE solver to form an end-to-end trainable framework. The superiority of the presented method is illustrated through the simulated and in-situ industrial robot datasets.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102851"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142417046","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Massive-Scale construction dataset synthesis through Stable Diffusion for Machine learning training","authors":"Sungkook Hong ,&nbsp;Byungjoo Choi ,&nbsp;Youngjib Ham ,&nbsp;JungHo Jeon ,&nbsp;Hyunsoo Kim","doi":"10.1016/j.aei.2024.102866","DOIUrl":"10.1016/j.aei.2024.102866","url":null,"abstract":"<div><div>Advancements of artificial intelligence (AI)-driven image generation provide opportunities to address a problem in machine learning applications that have suffered from a lack of domain-specific training data. This study explores the feasibility of employing synthesized images (SIs) generated through Stable Diffusion as training data for construction. This study aims to examine the potential of Stable Diffusion in construction, and the performance of convolutional neural network (CNN) models trained exclusively on SIs. A total of 82.01% of images synthesized are suitable for representing construction tasks. The CNN model trained on preprocessed SIs (with context-based labeling results) exhibited a classification accuracy of 89.09%. The CNN model trained solely on raw SIs (synthesized images without context-based labeling results) achieved a successful classification rate of 86.51% for the images. This study presents the viability of SIs as a training dataset and introduces context-based labeling through object detection techniques, enhancing the performance of estimation models.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102866"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142417048","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Human–machine cooperative decision-making and planning for automated vehicles using spatial projection of hand gestures","authors":"Yiran Zhang ,&nbsp;Zhongxu Hu ,&nbsp;Peng Hang ,&nbsp;Shanhe Lou ,&nbsp;Chen Lv","doi":"10.1016/j.aei.2024.102864","DOIUrl":"10.1016/j.aei.2024.102864","url":null,"abstract":"<div><div>Significant challenges in perception, prediction, and decision-making within self-driving systems remain inadequately addressed. Concurrently, the advancement of autonomous driving technologies reduces driver engagement, inadvertently eroding their proficiency. Integrating human cognitive flexibility and experiential insight with the machine’s precision and reliability offers a promising approach for the transitional phase towards fully automated driving. This study presents a human-machine collaboration approach to enhance the highly automated vehicles’ high-level flexibility and personalization attribute without the need for passengers’ prior driving experience. Firstly, we propose a tactical human–vehicle collaboration framework leveraging the hand-landmark extraction algorithm and augmented visual feedback. The proposed vision-based interface projects the gesture onto the ground and feeds it back to the driver through the augmented reality head-up display (AR-HUD) for intuitive interaction. The projection offers strategic decision-making guidance and planning recommendations for the vehicle. Utilizing these suggestions, the automation algorithm efficiently manages the remaining tasks, including collision avoidance and adherence to traffic regulations. This approach minimizes the driver’s engagement in routine driving tasks and negates the need for driving skills. Incorporating cooperative game theory, the methodology optimally balances personalization with system robustness. Finally, we compare our approach with conventional manual driving schemes that both can assist the self-driving car in avoiding unknown obstacles and reaching the personalized goal. Results demonstrate that the proposed decision-making and planning collaboration scheme significantly reduces human physical burdens without compromising driving performance and driver mental workloads.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102864"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142442731","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Fault diagnosis of mobile robot based on dual-graph convolutional network with prior fault knowledge","authors":"Longda Zhang ,&nbsp;Fengyu Zhou ,&nbsp;Peng Duan ,&nbsp;Xianfeng Yuan","doi":"10.1016/j.aei.2024.102865","DOIUrl":"10.1016/j.aei.2024.102865","url":null,"abstract":"<div><div>Effective integration of multi-sensor measurements is crucial for mobile robot fault diagnosis. However, in multi-sensor relationship modeling, existing methods often neglect the impact of different fault types and fail to consider the relations among data samples. To address these issues, a novel dual-graph convolutional network with prior fault knowledge (FKDGCN) is proposed. Specifically, we construct multi-sensor topological graphs based on prior fault knowledge, which effectively consider the impact of fault categories on sensor correlations. Subsequently, sample affinity graphs are constructed based on the temporal relationship and data similarity, and a sample correlation feature extraction module (SCFEM) is designed to capture the interdependence among data samples. Eventually, a novel dual-graph convolutional network is proposed to fuse multi-sample features and multi-sensor spatial–temporal features, in which more comprehensive fault information can be extracted. The effectiveness of FKDGCN is thoroughly validated on datasets collected from a real robot fault diagnosis test bench. Experimental results indicate that FKDGCN achieves outstanding diagnosis performance compared to state-of-the-art methods, with an average accuracy of over 98% on the balanced dataset and over 90% on two imbalanced datasets.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102865"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142446802","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}