Huang Zhang, Zili Wang, Shuyou Zhang, Lemiao Qiu, Yang Wang, Feifan Xiang, Zhiwei Pan, Linhao Zhu, Jianrong Tan
{"title":"Digital-Triplet: a new three entities digital-twin paradigm for equipment fault diagnosis","authors":"Huang Zhang, Zili Wang, Shuyou Zhang, Lemiao Qiu, Yang Wang, Feifan Xiang, Zhiwei Pan, Linhao Zhu, Jianrong Tan","doi":"10.1007/s10845-024-02471-7","DOIUrl":"https://doi.org/10.1007/s10845-024-02471-7","url":null,"abstract":"<p>Current equipment fault diagnosis faces challenges due to the difficulties in arranging sensors to collect effective data and obtaining diverse fault data for studying fault mechanisms. The lack of data results in disconnection between data from different spaces, posing a challenge to forming a closed loop of data and hindering the development of digital twin (DT) driven fault diagnosis (FD). To address these issues, a new DT paradigm Digital-Triplet is proposed. This paradigm comprises three entities: a physical entity, a semi-physical entity, and a virtual entity. A semi-physical entity is created by implementing the \"six-D\" process on the physical entity. A new six dimensional structure is formed through the addition of the semi-physical entity. The new structure streamlines the construction of fault datasets, enhances sensor data acquisition, and tightly links different data spaces, thereby promoting the application of DT in equipment FD. Subsequently, the elevator is selected as a case study to illustrate the Digital-Triplet framework in detail. The results demonstrate that the Digital-Triplet framework can effectively expand the fault dataset and improve data collection efficiency through optimized sensor placement, thereby promoting fault diagnosis.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"28 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-08-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142213335","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jan-Philipp Kaiser, Jonas Gäbele, Dominik Koch, Jonas Schmid, Florian Stamer, Gisela Lanza
{"title":"Adaptive acquisition planning for visual inspection in remanufacturing using reinforcement learning","authors":"Jan-Philipp Kaiser, Jonas Gäbele, Dominik Koch, Jonas Schmid, Florian Stamer, Gisela Lanza","doi":"10.1007/s10845-024-02478-0","DOIUrl":"https://doi.org/10.1007/s10845-024-02478-0","url":null,"abstract":"<p>In remanufacturing, humans perform visual inspection tasks manually. In doing so, human inspectors implicitly solve variants of visual acquisition planning problems. Nowadays, solutions to these problems are computed based on the object geometry of the object to be inspected. In remanufacturing, however, there are often many product variants, and the existence of geometric object models cannot be assumed. This makes it difficult to plan and solve visual acquisition planning problems for the automated execution of visual inspection tasks. Reinforcement learning offers the possibility of learning and reproducing human inspection behavior and solving the visual inspection problem, even for problems in which no object geometry is available. To investigate reinforcement learning as a solution, a simple simulation environment is developed, allowing the execution of reproducible and controllable experiments. Different reinforcement learning agent modeling alternatives are developed and compared for solving the derived visual planning problems. The results of this work show that reinforcement learning agents can solve the derived visual planning problems in use cases without available object geometry by using domain-specific prior knowledge. Our proposed framework is available open source under the following link: https://github.com/Jarrypho/View-Planning-Simulation.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"77 3 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-08-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142213336","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Random convolution layer: an auxiliary method to improve fault diagnosis performance","authors":"Zhiqian Zhao, Runchao Zhao, Yinghou Jiao","doi":"10.1007/s10845-024-02458-4","DOIUrl":"https://doi.org/10.1007/s10845-024-02458-4","url":null,"abstract":"<p>In real industry, it is often difficult to obtain large-scale labeled data. Existing Convolutional Neural Network (CNN)-based fault diagnosis methods often struggle to achieve accurate diagnoses of machine conditions due to the scarcity of labeled data, hindering the ability of models to develop strong inductive biases. We propose a plug-and-play auxiliary method, random convolution layer (RCL), to improve the generalization performance of the fault diagnosis models. This method delves into the fundamental commonalities across diverse tasks and varying network structures, thereby enhancing the diversity of samples to establish a more robust source domain environment. The RCL preserves the dimensional nature of the data in the time domain while randomly altering the kernel sizes during convolution operations, thus generating new data without compromising global information. During the training process, the newly generated data is mixed with the original data and fed into the fault diagnosis model. RCL is incorporated as a module into the inputs of different fault diagnosis models, and its effectiveness is validated on three public datasets as well as a self-built testbed. The results show that the present auxiliary method improves the domain generalization performance of the baselines, and can improve the accuracy of the corresponding fault diagnosis models. Our code is available at https://github.com/zhiqan/Random-convolution-layer.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"5 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142213337","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pradeep Kundu, Ashish K. Darpe, Makarand S. Kulkarni
{"title":"Development of data-driven, physics-based, and hybrid prognosis frameworks: a case study for gear remaining useful life prediction","authors":"Pradeep Kundu, Ashish K. Darpe, Makarand S. Kulkarni","doi":"10.1007/s10845-024-02477-1","DOIUrl":"https://doi.org/10.1007/s10845-024-02477-1","url":null,"abstract":"<p>Data-driven, physics-based, and hybrid prognosis frameworks can be developed to estimate remaining useful life, depending on the availability of condition monitoring sensor data and physics-governing equations. No systematic study is available that shows the comparative performance of these frameworks. The present study, for the first time, attempts to show how these three frameworks can be developed under different scenarios and assumptions. The data-driven prognosis framework is developed using an accelerometer signal and an Artificial Intelligence-based random forest regression (RFR) model. A pit growth model inspired by the Paris crack growth law has been used for physics-based prognosis framework development. In this framework, sensor data is needed to know the gear’s current health status, as the prognosis framework can't be developed purely on physics. A hybrid prognosis framework is developed using two alternate approaches: one in which current health status is obtained directly from a visual inspection camera and the other in which this status is indirectly inferred from the accelerometer sensor data. In each case, the RUL prediction is made using a physics-based pit growth model coupled with the current health status obtained from either of the two approaches mentioned. To enhance the prediction accuracy, Bayesian inference is used to update the physics-based pit growth model parameters in both hybrid frameworks. Data obtained from five run-to-failure experiments performed on a specially designed gearbox test setup are used to show the comparative performance of these frameworks. The strengths and weaknesses of each of the frameworks are discussed based on the type of data requirement, model definition, parameter estimation, and prediction error.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"20 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142213338","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Aitha Sudheer Kumar, Ankit Agarwal, Vinita Gangaram Jansari, K. A. Desai, Chiranjoy Chattopadhyay, Laine Mears
{"title":"HG-XAI: human-guided tool wear identification approach through augmentation of explainable artificial intelligence with machine vision","authors":"Aitha Sudheer Kumar, Ankit Agarwal, Vinita Gangaram Jansari, K. A. Desai, Chiranjoy Chattopadhyay, Laine Mears","doi":"10.1007/s10845-024-02476-2","DOIUrl":"https://doi.org/10.1007/s10845-024-02476-2","url":null,"abstract":"<p>Identifying tool wear state is essential for machine operators as it assists in informed decisions for timely tool replacement and subsequent machining operations. As each wear state corresponds to a unique mitigation strategy, timely identification is vital while implementing solutions to minimize tool wear. The paper presents a novel Human Guided-eXplainable Artificial Intelligence (HG-XAI) approach for identifying the tool wear state by integrating human intelligence and eXplainable AI with a pre-trained Convolutional Neural Network (CNN), Efficient-Net-b0 model. The tool wear states were identified based on different wear mechanisms during the machining of IN718. The study considers four distinct tool wear states, i.e., Flank, Flank+BUE, Flank+Face, and Chipping, representing abrasion, adhesion, diffusion, and fracture wear mechanisms. The image-based datasets were created to depict various tool wear states by machining IN718 at varying surface speeds. The effectiveness of the proposed HG-XAI approach was evaluated by comparing its prediction accuracy with a standalone Efficient-Net-b0 model lacking human intelligence and XAI. Further, the scalability of the HG-XAI approach was examined by predicting wear states from images acquired at different cutting parameters. The results from the present study showed that the HG-XAI approach can predict the tool wear state with an accuracy of 93.08% and is scalable to variations in cutting conditions. Also, the proposed approach can be extended while developing vision-based on-machine tool wear monitoring systems.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"14 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-08-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142213339","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A multi-task effectiveness metric and an adaptive co-training method for enhancing learning performance with few samples","authors":"Xiaoyao Wang, Fuzhou Du, Delong Zhao, Chang Liu","doi":"10.1007/s10845-024-02475-3","DOIUrl":"https://doi.org/10.1007/s10845-024-02475-3","url":null,"abstract":"<p>The integration of deep learning (DL) into vision inspection methods is increasingly recognized as a valuable approach to substantially enhance the adaptability and robustness. However, it is well known that high-performance neural networks typically require large training datasets with high-quality manual annotations, which are difficult to obtain in many manufacturing processes. To enhance the performance of DL methods for vision task with few samples, this paper proposes a novel metric called Effectiveness of Auxiliary Task (EAT) and presents a multi-task learning approach utilizing this metric for selecting effective auxiliary task branch and adaptive co-training them with main tasks. Experiments conducted on two vision tasks with few samples show that the proposed approach effectively eliminates ineffective task branches and enhances the contribution of the selected tasks to the main task: reducing the average normalized pixel error from 0.0613 to 0.0143 in pose key-points detection and elevating the Intersection over Union (IoU) from 0.6383 to 0.6921 in surface defect segmentation. Remarkably, these enhancements are achieved without necessitating additional manual labeling efforts.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"193 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141934214","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Managing product-inherent constraints with artificial intelligence: production control for time constraints in semiconductor manufacturing","authors":"Marvin Carl May, Jan Oberst, Gisela Lanza","doi":"10.1007/s10845-024-02472-6","DOIUrl":"https://doi.org/10.1007/s10845-024-02472-6","url":null,"abstract":"<p>Continuous product individualization and customization led to the advent of lot size one in production and ultimately to product-inherent uniqueness. As complexities in individualization and processes grow, production systems need to adapt to unique, product-inherent constraints by advancing production control beyond predictive, rigid schedules. While complex processes, production systems and production constraints are not a novelty per se, modern production control approaches fall short of simultaneously regarding the flexibility of complex job shops and product unique constraints imposed on production control. To close this gap, this paper develops a novel, data driven, artificial intelligence based production control approach for complex job shops. For this purpose, product-inherent constraints are resolved by restricting the solution space of the production control according to a prediction based decision model. The approach validation is performed in a real semiconductor fab as a job shop that includes transitional time constraints as product-inherent constraints. Not violating these time constraints is essential to avoid scrap and similarly increase quality-based yield. To that end, transition times are forecasted and the adherence to these product-inherent constraints is evaluated based on one-sided prediction intervals and point estimators. The inclusion of product-inherent constraints leads to significant adherence improvements in the production system as indicated in the real-world semiconductor manufacturing case study and, hence, contributes a novel, data driven approach for production control. As a conclusion, the ability to avoid a large majority of violations of time constraints shows the approaches effectiveness and the future requirement to more accurately integrate such product-inherent constraints into production control.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"36 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-08-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141886637","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Qibin Wang, Linyang Yu, Liang Hao, Shengkang Yang, Tao Zhou, Wanghui Ji
{"title":"An adaptive transfer fault detection method for rotary machine with multi-sensor information fusion","authors":"Qibin Wang, Linyang Yu, Liang Hao, Shengkang Yang, Tao Zhou, Wanghui Ji","doi":"10.1007/s10845-024-02469-1","DOIUrl":"https://doi.org/10.1007/s10845-024-02469-1","url":null,"abstract":"<p>Multi-sensor information fusion method has good performance in fault detection of rotary machine, in which each sensor information has made different contributions. The contribution of each sensor changes based on the working conditions of the machine, which can lead to a degradation in the performance of the transfer method when used in cross-domain mechanical fault detection. To solve this problem, an adaptive transfer fault detection method for rotary machine with multi-sensor information fusion is proposed. Firstly, multi-sensor data under different working conditions is collected, and features of different sensors are extracted by the corresponding deep learning model. Secondly, the multi-information interaction fusion network is designed to exchange sensor information and obtain fusion features. Then the fusion feature transfer model is proposed for cross-domain fault detection. Finally, the model is trained with the bearing dataset of the University of Paderborn. The results show that the transfer fault detection method with multi-sensor information fusion achieves state-of-the-art performances in cross-domain fault detection. It can adjust adaptively the contribution of each sensor information in the cross-domain fault detection.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"82 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141886643","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Francesco Lupi, Nelson Freitas, Miguel Arvana, Andre Dionisio Rocha, Antonio Maffei, José Barata, Michele Lanzetta
{"title":"Next-generation Vision Inspection Systems: a pipeline from 3D model to ReCo file","authors":"Francesco Lupi, Nelson Freitas, Miguel Arvana, Andre Dionisio Rocha, Antonio Maffei, José Barata, Michele Lanzetta","doi":"10.1007/s10845-024-02456-6","DOIUrl":"https://doi.org/10.1007/s10845-024-02456-6","url":null,"abstract":"<p>This paper proposes and implements a novel pipeline for the self-reconfiguration of a flexible, reconfigurable, CAD-based, and autonomous Vision Inspection System (VIS), expanding upon the modular framework theoretically outlined in (Lupi, F., Maffei, A., & Lanzetta, M. (2024). CAD-based Autonomous Vision Inspection Systems. <i>Procedia Computer Science</i>, <i>232</i>, 2127–2136. https://doi.org/10.1016/J.PROCS.2024.02.033.). The pipeline automates the extraction and processing of inspection features manually incorporated by the designer into the Computer Aided Design (CAD) 3D model during the design stage, in accordance with Model Based Design (MBD) principles, which, in turn, facilitate virtuous approaches such as concurrent engineering and design for (Dfx), ultimately minimizing the time to market. The enriched CAD, containing inspection annotations (textual or dimensional) attached to geometrical entities, serving as the pipeline’s input, can be exported in a neutral file format, adhering to the Standard for Product Data Exchange (STEP) Application Protocol (AP)242, regardless of the modeling software used. The pipeline’s output is a Reconfiguration (ReCo) file, enabling the flexible hardware (e.g., robotic inspection cell) and software components of the VIS to be reconfigured via software (programmable). The main achievements of this work include: (i) demonstrating the feasibility of an end-to-end (i.e., CAD-to-ReCo file) pipeline that integrates the proposed software modules via Application Programming Interfaces (API)s, and (ii) formally defining the ReCo file. Experimental results from a demonstrative implementation enhance the clarity of the paper. The accuracy in defect detection achieved a 96% true positive rate and a 6% false positive rate, resulting in an overall accuracy of 94% and a precision of 88% across 72 quality inspection checks for six different inspection features of two product variants, each tested on six samples.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"68 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141880424","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Deep-learning based artificial intelligence tool for melt pools and defect segmentation","authors":"Amra Peles, Vincent C. Paquit, Ryan R. Dehoff","doi":"10.1007/s10845-024-02457-5","DOIUrl":"https://doi.org/10.1007/s10845-024-02457-5","url":null,"abstract":"<p>Accelerating fabrication of additively manufactured components with precise microstructures is important for quality and qualification of built parts, as well as for a fundamental understanding of process improvement. Accomplishing this requires fast and robust characterization of melt pool geometries and structural defects in images. This paper proposes a pragmatic approach based on implementation of deep learning models and self-consistent workflow that enable systematic segmentation of defects and melt pools in optical images. Deep learning is based on an image-to-image translation–conditional generative adversarial neural network architecture. An artificial intelligence (AI) tool based on this deep learning model enables fast and incrementally more accurate predictions of the prevalent geometric features, including melt pool boundaries and printing-induced structural defects. We present statistical analysis of geometric features that is enabled by the AI tool, showing strong spatial correlation of defects and the melt pool boundaries. The correlations of widths and heights of melt pools with dataset processing parameters show the highest sensitivity to thermal influences resulting from laser passes in adjacent and subsequent layer passes. The presented models and tools are demonstrated on the aluminum alloy and datasets produced with different sets of processing parameters. However, they have universal quality and could easily be adapted to different material compositions. The method can be easily generalized to microstructural characterizations other than optical microscopy.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"8 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-07-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141870826","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}