{"title":"Augmenting human-guided progressive learning with machine vision systems for robust surface defect detection","authors":"Swarit Anand Singh, Sahil J Choudhari, K.A. Desai","doi":"10.1016/j.aei.2024.102906","DOIUrl":"10.1016/j.aei.2024.102906","url":null,"abstract":"<div><div>Machine vision systems commonly utilize Convolutional Neural Networks (CNNs) for in-line surface defect detection of manufactured components. The prediction abilities of vision-based inspection systems deteriorate with time as the defect detection model trained on fixed image datasets fails to accommodate deviations. This paper proposes a human-guided progressive learning approach that systematically imparts learning of new features to the CNN-powered vision-based defect detection system. The approach augments the surface defect detection model with human intelligence, using an intuitive user interface to address model drift. The human expert monitors the trained model performance under specific conditions leading to the change of characteristics during implementation, identifies misclassifications, and initiates re-training. The algorithm accumulates misclassified data till a pre-defined threshold level is reached or a human expert terminates inspection. The misclassified results merge with the original datasets for progressive re-training using a strategy similar to the base model development. The present work utilizes pre-trained CNN Efficientnet-b0 to develop the surface defect detection model for tapered roller inspection through transfer learning. It is concluded that the progressive re-training improves defect detection performance and reduces misclassifications. The Matthews Correlation Coefficient (MCC) score, derived from the confusion matrix, showed improvement from 0.6 to 0.82 after four iterations. A cross-model benchmarking study is also performed to show the versatility of the proposed approach. The present work demonstrated that the human-guided progressive learning approach can provide adaptability to vision-based surface defect detection utilizing deep learning algorithms and enhance system performance during real-time implementation.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102906"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142532007","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yixiang Huang , Kaiwen Zhang , Pengcheng Xia , Zhilin Wang , Yanming Li , Chengliang Liu
{"title":"Cross-attentional subdomain adaptation with selective knowledge distillation for motor fault diagnosis under variable working conditions","authors":"Yixiang Huang , Kaiwen Zhang , Pengcheng Xia , Zhilin Wang , Yanming Li , Chengliang Liu","doi":"10.1016/j.aei.2024.102948","DOIUrl":"10.1016/j.aei.2024.102948","url":null,"abstract":"<div><div>Motor fault diagnosis under variable working conditions is an open challenge for practical application. Domain adaptation has been explored for reducing feature distribution discrepancy across working conditions. However, existing methods overlook the relations and the domain-related features among individual sample pairs across different domains, and the quality of pseudo labels significantly limits the subdomain adaptation performance. To tackle these limitations, a cross-attentional subdomain adaptation (CroAttSA) method with clustering-based selective knowledge distillation for motor fault diagnosis under variable working conditions is proposed. A triple-branch transformer with self-attention and cross-domain-attention is designed for domain-specific and domain-correlated feature extraction. Additionally, a correlated local maximum mean discrepancy (CLMMD) loss is introduced for more fine-grained and fault-related subdomain adaptation. A clustering-based selective knowledge distillation strategy is also proposed to improve the quality of the pseudo labels for enhanced model performance. Extensive experiments on motor fault diagnosis under variable loads and rotating speeds are conducted, and the comparison and ablation study results have verified the model effectiveness.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102948"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142701265","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yuxiao Wang , Hongming Cai , Bingqing Shen , Pan Hu , Han Yu , Lihong Jiang
{"title":"CGCI: Cross-granularity Causal Inference framework for engineering Change Propagation Analysis","authors":"Yuxiao Wang , Hongming Cai , Bingqing Shen , Pan Hu , Han Yu , Lihong Jiang","doi":"10.1016/j.aei.2024.102918","DOIUrl":"10.1016/j.aei.2024.102918","url":null,"abstract":"<div><div>In the dynamic landscape of large-scale and intricate product development, the constant generation and accumulation of configuration data, influenced by factors such as evolving demands and version alterations, exhibit inter-domain and inter-level characteristics. This complexity presents formidable challenges to the management of controlled changes. Central to effective change management is Change Propagation Analysis (CPA), particularly in accurately predicting the potential impacts on affected items. However, conventional CPA methods are insufficient for addressing the challenge of cross-domain, cross-level inference. Therefore, we propose a Cross-granularity Causal Inference Framework (CGCI) tailored for CPA. This framework leverages the diffusion and attenuation of influence, enabling efficient identification of potential configuration items. To assess the feasibility of CGCI, a dataset is constructed using raw industrial configuration data and conducted a comprehensive case study on aircraft configuration change control. The results of our comparative analysis show that CGCI is effective in addressing multi-granularity and multi-hop inference problems, with more comprehensive consideration and less inference overhead in the multi-granularity case.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102918"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142701268","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Longlong He , Jiani Gao , Jiewu Leng , Yue Wu , Kai Ding , Lin Ma , Jie Liu , Duc Truong Pham
{"title":"Disassembly sequence planning of equipment decommissioning for industry 5.0: Prospects and Retrospects","authors":"Longlong He , Jiani Gao , Jiewu Leng , Yue Wu , Kai Ding , Lin Ma , Jie Liu , Duc Truong Pham","doi":"10.1016/j.aei.2024.102939","DOIUrl":"10.1016/j.aei.2024.102939","url":null,"abstract":"<div><div>With the advent of Industry 5.0, the complexity and variety involved in disassembling decommissioned equipment have increased significantly, underscoring the growing importance of disassembly sequence planning (DSP) for resource recovery and reuse. Industry 5.0 emphasizes human-centricity, resilience, and sustainable development, raising new challenges and higher standards for DSP technologies and methods. While previous studies have highlighted the need to study DSP in the context of Industry 5.0, focusing on leveraging technological advancements to optimize the disassembly process, our work takes a different approach. We emphasize the integration of intelligent systems and human–machine collaboration to provide comprehensive solutions, from constructing information models to optimizing sequence algorithms, while also exploring emerging research directions to address the demands of this new era. In order to address the evolving challenges presented by Industry 5.0, this study seeks to reevaluate the pivotal role of DSP in the domain of retired equipment. It also intends to conduct a thorough investigation of DSP from the perspectives of humanism, resilience, and sustainability. By assessing the applicability of existing DSP approaches in the Industry 5.0 landscape, there is a specific focus on the integration of big data analytics and intelligent algorithms to enhance disassembly efficiency, optimize resource allocation, and achieve environmentally sustainable development goals. The research reveals certain limitations in the current state of DSP, namely in terms of intelligence, flexibility, and sustainability. For Industry 5.0, DSP should holistically consider human factors, robustness, and sustainability. Adopting these approaches enhances disassembly efficiency, optimizes resource utilization, mitigates environmental impact, and promotes the achievement of sustainable development goals.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102939"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142701270","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ziliang Wang , Wei Guo , Hongyu Shao , Lei Wang , Zhixing Chang , Yuanrong Zhang , Zhenghong Liu
{"title":"From technology opportunities to solutions generation via patent analysis: Application of machine learning-based link prediction","authors":"Ziliang Wang , Wei Guo , Hongyu Shao , Lei Wang , Zhixing Chang , Yuanrong Zhang , Zhenghong Liu","doi":"10.1016/j.aei.2024.102944","DOIUrl":"10.1016/j.aei.2024.102944","url":null,"abstract":"<div><div>Technology convergence represents a significant mode of technological innovation that is widely prevalent across various industries. This innovative approach integrates multiple technologies to develop new integrated solutions, thereby fostering a competitive advantage for enterprises. Anticipating future potential technology convergence is of paramount importance for businesses. However, previous research has predominantly relied on the topological information of convergence networks, overlooking the nodal attributes and inter-nodal relationships that have an impact on the emergence of technology convergence. To enhance existing studies, this paper employs three types of features: node attributes and inter-node relationships based on the drivers of technology convergence, along with link prediction similarity indices. Additionally, we utilize Graph Convolutional Neural Network (GCN) for node embedding to leverage node attributes. Machine learning models are utilized for link prediction based on these features to identify potential technology opportunities. To guide research and development (R&D) efforts, we recommend high-value patents for each node using entropy weighting across five metrics that objectively quantify patent value, and transform patent abstracts into vectors using Doc2Vec. Patents with high similarity in abstract text between nodes are utilized to extract technical solutions and fuse ideas for technology convergence. A case study is conducted within the autonomous driving industry, leveraging comprehensive information including node attributes, inter-node relationships, and topology-based similarities to identify technology opportunities and guide the generation of R&D ideas through the convergence of technical solutions.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102944"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142659054","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Interpretable large-scale belief rule base for complex industrial systems modeling with expert knowledge and limited data","authors":"Zheng Lian, Zhijie Zhou, Changhua Hu, Zhichao Feng, Pengyun Ning, Zhichao Ming","doi":"10.1016/j.aei.2024.102852","DOIUrl":"10.1016/j.aei.2024.102852","url":null,"abstract":"<div><div>Complex system modeling technology is a hot topic. Nowadays, many complex industrial systems present three characteristics: multiple input indicators, limited data and interpretability requirements. With good interpretability, belief rule base (BRB) serves as an efficient tool for modeling complex systems. However, as the number of input indicators of industrial systems increases, BRB suffers from the combinatorial explosion problem, which makes it hard to generate large-scale BRB and optimize it while maintaining its interpretability. For this purpose, an interpretable large-scale BRB is proposed for complex systems with limited data, where expert knowledge can be utilized effectively. First, a framework for generating an initial large-scale BRB using expert knowledge and limited data is developed, including the determination of attribute weight, basic belief degree and rule weight. Afterwards, a new parameter optimization model is designed to reduce the burden of parameter optimization and maintain the interpretability of BRB, where the Adaptive Moment Estimation (Adam) algorithm is adopted to further improve the efficiency of large-scale parameter optimization. Finally, a health assessment case of an inertial navigation system (INS) verifies the proposed method.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102852"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142417210","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Smeta-LU: A self-supervised meta-learning fault diagnosis method for rotating machinery based on label updating","authors":"Zhiqian Zhao , Yinghou Jiao , Yeyin Xu , Zhaobo Chen , Runchao Zhao","doi":"10.1016/j.aei.2024.102875","DOIUrl":"10.1016/j.aei.2024.102875","url":null,"abstract":"<div><div>During operation of rotating machinery, collecting high-quality labeled fault samples is difficult, and the corresponding data annotation is time consuming and costly. Therefore, developing novel intelligent diagnostic methods which can extract key information from massive fault data without labeling is of great significance. In this regard, a self-supervised meta-learning fault diagnosis method for rotating machinery based on label updating, called Smeta-LU, is proposed. It eliminates the pre-training phase and generates meta-tasks directly without labeling information during training. A two-branch framework in Smeta-LU is developed using a contrastive learning approach, which involves the application of a dynamic dictionary to construct samples for one branch, represented by an online encoder. The other branch utilizes the parameters of the former to obtain a target encoder through exponential moving average. To dynamically construct diverse meta-tasks during the meta-training process, each sample in the current batch is treated as a query set, while the support set is selected from queues to construct few-shot tasks, thereby generating a larger pool of candidates. The fault diagnosis task is completed by assigning the label matrix with an optimal transport algorithm and identifying the shots closest to each of the prototype centers. Additionally, the iterative properties of the momentum network and dynamic dictionary are implemented for label updating. The outcomes of two validation experiments demonstrate the superiority and scalability of our self-supervised meta-learning approach compared with conventional supervised meta-learning techniques. Better performance in identifying new fine-grained fault categories is also exhibited during our research.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102875"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142529669","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yuntae Jeon , Dai Quoc Tran , Almo Senja Kulinan , Taeheon Kim , Minsoo Park , Seunghee Park
{"title":"Vision-based motion prediction for construction workers safety in real-time multi-camera system","authors":"Yuntae Jeon , Dai Quoc Tran , Almo Senja Kulinan , Taeheon Kim , Minsoo Park , Seunghee Park","doi":"10.1016/j.aei.2024.102898","DOIUrl":"10.1016/j.aei.2024.102898","url":null,"abstract":"<div><div>Ensuring worker safety on dynamic construction sites is a significant challenge, especially as it is crucial to immediately identify potential hazards and warn workers. Existing computer vision-based motion prediction methods often overlook the false negative issue caused by the noisy environments of construction sites, and treat tracking and trajectory prediction as disconnected processes. This study introduces MPSORT, a method that suggests trajectory prediction-based tracking with trajectory interpolation for vision-based automated safety monitoring. The proposed method predicts the future movements of construction workers and vehicles using multiple CCTV cameras, and localizes these predictions onto the construction site’s bird’s eye view (BEV) map. This enables to send the real-time warnings to workers in danger, preventing accidents such as collision, fall, and getting stuck. We evaluated the performance of our method in both object tracking and trajectory prediction tasks on dataset from multiple CCTV cameras on construction sites. The object tracking results show an approximate 10% increase in the number of tracked objects, and the trajectory prediction results indicate an ADE of 7.138 and an FDE of 12.493, reflecting improvements of more than 5% and 2% in ADE and FDE, respectively, compared to previous methods. Overall, these findings are significant for minimizing accidents and enhancing safety and efficiency on construction sites.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102898"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142578040","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zeqiang Zhang , Wei Liang , Dan Ji , Yanqing Zeng , Yu Zhang , Yan Li , Lixia Zhu
{"title":"Mixed integer programming and multi-objective enhanced differential evolution algorithm for human–robot responsive collaborative disassembly in remanufacturing system","authors":"Zeqiang Zhang , Wei Liang , Dan Ji , Yanqing Zeng , Yu Zhang , Yan Li , Lixia Zhu","doi":"10.1016/j.aei.2024.102895","DOIUrl":"10.1016/j.aei.2024.102895","url":null,"abstract":"<div><div>The recycling of waste products is essential for resource reuse. However, turning operation direction causes significant fatigue to operators handling end-of-life (EoL) products, consequently degrading the recycling efficiency. Accordingly, this study employs responsive collaboration robots to aid operators in turning the operation direction of disassembled products. To solve the human-robot responsive collaboration disassembly line balancing problem (HRRC-DLBP), a mixed integer programming (MIP) model is constructed, and a decoding mechanism is designed in this study. Additionally, a multi-objective enhanced differential evolution algorithm (MEDE) in which the decoding mechanism is incorporated is devised and applied to solve the HRRC-DLBP. The MEDE algorithm is validated by comparing its solution results with those of the MIP model. Finally, the MEDE is used to optimise the EoL printer case for the HRRC-DLBP and the disassembly line balancing problem in which the operation direction is turned by humans (H-DLBP). The optimisation results show that the recycling of EoL products is more efficient using the HRRC-DLBP than employing the H-DLBP.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102895"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142532181","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lan Bo , Tiezhu Zhang , Hongxin Zhang , Jichao Hong , Mingjie Liu , Caihong Zhang , Benyou Liu
{"title":"3D UAV path planning in unknown environment: A transfer reinforcement learning method based on low-rank adaption","authors":"Lan Bo , Tiezhu Zhang , Hongxin Zhang , Jichao Hong , Mingjie Liu , Caihong Zhang , Benyou Liu","doi":"10.1016/j.aei.2024.102920","DOIUrl":"10.1016/j.aei.2024.102920","url":null,"abstract":"<div><div>The increasing number of application scenarios necessitate unmanned aerial vehicles to possess the capability of autonomous obstacle avoidance and navigation in unknown environments, representing a crucial direction for its development. Path planning plays a crucial role in this process. Path planning aims to design efficient and safe navigation paths for UAVs, thereby significantly reducing energy consumption and time spent while improving equipment adaptability to the environment. Firstly, we employ the deep reinforcement learning algorithm to train the agent on randomly changing maps, enabling it to possess both generalization capabilities and active obstacle avoidance skills. Secondly, a novel framework combining transfer reinforcement learning is proposed. It establishes the pre-trained model and utilizes the enhanced low-rank adaptive algorithm to transfer it into formal training, thereby incorporating prior knowledge and improving the efficacy of formal training. Finally, a novel method of sample abundance is proposed to reuse the experience pool stored in the pre-trained model and further increase the generalization capability of the agent, thereby significantly improving its success rate. The proposed algorithm efficiently uses both the pre-trained model and the experience pool. In practical applications, the pre-trained model can be acquired by training on a limited dataset to endow the agent with autonomous obstacle avoidance capabilities. In formal training, numerous random samples are established to simulate unfamiliar environmental terrains. After rapid training, the agent achieves a success rate of 95% in the test set and demonstrates exceptional performance in smoothness and path length.</div></div>","PeriodicalId":50941,"journal":{"name":"Advanced Engineering Informatics","volume":"62 ","pages":"Article 102920"},"PeriodicalIF":8.0,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142572916","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}