Journal of Intelligent Manufacturing最新文献

{"title":"Multi-modal background-aware for defect semantic segmentation with limited data","authors":"Dexing Shan, Yunzhou Zhang, Shitong Liu","doi":"10.1007/s10845-024-02373-8","DOIUrl":"https://doi.org/10.1007/s10845-024-02373-8","url":null,"abstract":"<p>Visual defect detection is widely used in intelligent manufacturing to achieve intelligent detection of product quality. Two main challenges remain in industrial applications. One is the scarcity of defect samples and the other is the weak texture variation of industrial defects. The above problems lead to the application of RGB image-based industrial defect segmentation. To this end, we propose a multi-modal background-aware network (MMBA-Net) for few-shot defect (2D+3D) segmentation with limited data, which can segment texture and structural defects in unseen and seen domains (objects). To synthesize the perception capabilities of different imaging conditions, MMBA-Net exploits the point cloud to provide spatial information for the RGB images. Furthermore, we found that background regions are perceptually consistent within an industrial image, which can be leveraged to discriminate between foreground and background regions. To implement this idea, we model correlation learning between multi-modal query samples and multi-modal normal (defect-free) samples as an optimal transport problem, establishing robust multi-modal background correlations between query and normal samples across different modalities. Experiments were conducted on real-world industrial products and food datasets, demonstrating that the proposed method can perform effective base learning and meta-learning on a small number of defective samples (approximately 15–25 defective training samples) to achieve effective segmentation of defects in the seen and unseen domains.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"20 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-05-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141063323","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Defining a feature-level digital twin process model by extracting machining features from MBD models for intelligent process planning","authors":"Jingjing Li, Guanghui Zhou, Chao Zhang, Junsheng Hu, Fengtian Chang, Andrea Matta","doi":"10.1007/s10845-024-02406-2","DOIUrl":"https://doi.org/10.1007/s10845-024-02406-2","url":null,"abstract":"<p>The booming development of emerging technologies and their integration in process planning provide new opportunities for solving the problems in traditional trial-and-error process planning. Combining digital twin with 3D computer vision, this paper defines a novel feature-level digital twin process model (FL-DTPM) by extracting machining features from model-based definition models. Firstly, a multi-dimensional FL-DTPM framework is defined by fusing on-site data, quality information, and process knowledge, where the synergistic mechanism of its virtual and physical processes is revealed. Then, 3D computer vision-enabled machining features extraction method is embedded into the FL-DTPM framework to support the reuse of process knowledge, which involves the procedures of data pre-processing, semantic segmentation, and instance segmentation. Finally, the effectiveness of the proposed features extraction method is verified and the application of FL-DTPM in machining process is presented. Oriented to the impeller process planning, a prototype of FL-DTPM is constructed to explore the potential application scenarios of the proposed method in intelligent process planning, which could provide insights into the industrial implementation of FL-DTPM for aerospace manufacturing enterprises.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"94 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-05-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140928619","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A novel method based on deep learning algorithms for material deformation rate detection","authors":"Selim Özdem, İlhami Muharrem Orak","doi":"10.1007/s10845-024-02409-z","DOIUrl":"https://doi.org/10.1007/s10845-024-02409-z","url":null,"abstract":"<p>Given the significant influence of microstructural characteristics on a material’s mechanical, physical, and chemical properties, this study posits that the deformation rate of structural steel S235-JR can be precisely determined by analyzing changes in its microstructure. Utilizing advanced artificial intelligence techniques, microstructure images of S235-JR were systematically analyzed to establish a correlation with the material’s lifespan. The steel was categorized into five classes and subjected to varying deformation rates through laboratory tensile tests. Post-deformation, the specimens underwent metallographic procedures to obtain microstructure images via an light optical microscope (LOM). A dataset comprising 10000 images was introduced and validated using K-Fold cross-validation. This research utilized deep learning (DL) architectures ResNet50, ResNet101, ResNet152, VGG16, and VGG19 through transfer learning to train and classify images containing deformation information. The effectiveness of these models was meticulously compared using a suite of metrics including Accuracy, F1-score, Recall, and Precision to determine their classification success. The classification accuracy was compared across the test data, with ResNet50 achieving the highest accuracy of 98.45%. This study contributes a five-class dataset of labeled images to the literature, offering a new resource for future research in material science and engineering.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"22 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-05-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140928477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"MSOA: A modular service-oriented architecture to integrate mobile manipulators as cyber-physical systems","authors":"Nooshin Ghodsian, Khaled Benfriha, Adel Olabi, Varun Gopinath, Esma Talhi, Lucas Hof, Aurélien Arnou","doi":"10.1007/s10845-024-02404-4","DOIUrl":"https://doi.org/10.1007/s10845-024-02404-4","url":null,"abstract":"<p>In the evolving landscape of the fourth industrial revolution, the integration of cyber-physical systems (CPSs) into industrial manufacturing, particularly focusing on autonomous mobile manipulators (MMs), is examined. A comprehensive framework is proposed for embedding MMs into existing production systems, addressing the burgeoning need for flexibility and adaptability in contemporary manufacturing. At the heart of this framework is the development of a modular service-oriented architecture, characterized by adaptive decentralization. This approach prioritizes real-time interoperability and leverages virtual capabilities, which is crucial for the effective integration of MMs as CPSs. The framework is designed to not only accommodate the operational complexities of MMs but also ensure their seamless alignment with existing production control systems. The practical application of this framework is demonstrated at the Platform 4.0 research production line at Arts et Métiers. An MM named MoMa, developed by OMRON Company, was integrated into the system. This application highlighted the framework’s capacity to significantly enhance the production system's flexibility, autonomy, and efficiency. Managed by the manufacturing execution system (MES), the successful integration of MoMa exemplifies the framework's potential to transform manufacturing processes in alignment with the principles of Industry 4.0.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"123 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-05-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140928840","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Selecting subsets of source data for transfer learning with applications in metal additive manufacturing","authors":"Yifan Tang, Mostafa Rahmani Dehaghani, Pouyan Sajadi, G. Gary Wang","doi":"10.1007/s10845-024-02402-6","DOIUrl":"https://doi.org/10.1007/s10845-024-02402-6","url":null,"abstract":"<p>Considering data insufficiency in metal additive manufacturing (AM), transfer learning (TL) has been adopted to extract knowledge from source domains (e.g., completed printings) to improve the modeling performance in target domains (e.g., new printings). Current applications use all accessible source data directly in TL with no regard to the similarity between source and target data. This paper proposes a systematic method to find appropriate subsets of source data based on similarities between the source and limited target datasets. Such similarity is characterized by the spatial and model distance metrics. A Pareto frontier-based source data selection method is developed, where the source data located on the Pareto frontier defined by two similarity distance metrics are selected iteratively. This method is integrated into an instance-based TL method (decision tree regression model) and a model-based TL method (fine-tuned artificial neural network). Both models are then tested on several regression tasks in metal AM. Comparison results demonstrate that (1) the source data selection method is general and supports integration with various TL methods and distance metrics, (2) compared with using all source data, the proposed method can find a subset of source data from the same domain with better TL performance in metal AM regression tasks involving different processes and machines, and (3) when multiple source domains exist, the source data selection method could find the subset from one source domain to obtain comparable or better TL performance than the model constructed using data from all source domains.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"3 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-05-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140928604","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Remaining useful lifetime prediction for milling blades using a fused data prediction model (FDPM)","authors":"Teemu Mäkiaho, Jouko Laitinen, Mikael Nuutila, Kari T. Koskinen","doi":"10.1007/s10845-024-02398-z","DOIUrl":"https://doi.org/10.1007/s10845-024-02398-z","url":null,"abstract":"<p>In various industry sectors, predicting the real-life availability of milling applications poses a significant challenge. This challenge arises from the need to prevent inefficient blade resource utilization and the risk of machine breakdowns due to natural wear. To ensure timely and accurate adjustments to milling processes based on the machine's cutting blade condition without disrupting ongoing production, we introduce the Fused Data Prediction Model (FDPM), a novel temporal hybrid prediction model. The FDPM combines the static and dynamic features of the machines to generate simulated outputs, including average cutting force, material removal rate, and peripheral milling machine torque. These outputs are correlated with real blade wear measurements, creating a simulation model that provides insights into predicting the wear progression in the machine when associated with real machine operational parameters. The FDPM also considers data preprocessing, reducing the dimensional space to an advanced recurrent neural network prediction algorithm for forecasting blade wear levels in milling. The validation of the physics-based simulation model indicates the highest fidelity in replicating wear progression with the average cutting force variable, demonstrating an average relative error of 2.38% when compared to the measured mean of rake wear during the milling cycle. These findings illustrate the effectiveness of the FDPM approach, showcasing an impressive prediction accuracy exceeding 93% when the model is trained with only 50% of the available data. These results highlight the potential of the FDPM model as a robust and versatile method for assessing wear levels in milling operations precisely, without disrupting ongoing production.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"48 20 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-05-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140928608","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Generative AI approach to improve in-situ vision tool wear monitoring with scarce data","authors":"Alberto Garcia-Perez, Maria Jose Gomez-Silva, Arturo de la Escalera-Hueso","doi":"10.1007/s10845-024-02379-2","DOIUrl":"https://doi.org/10.1007/s10845-024-02379-2","url":null,"abstract":"<p>Most aerospace turbine casings are mechanised using a vertical lathe. This paper presents a tool wear monitoring system using computer vision that analyses tool inserts once that the machining process has been completed. By installing a camera in the robot magazine room and a tool cleaning device to remove chips and cooling residuals, a neat tool image can be acquired. A subsequent Deep Learning (DL) model classifies the tool as acceptable or not, avoiding the drawbacks of alternative computer vision algorithms based on edges, dedicated features etc. Such model was trained with a significantly reduced number of images, in order to minimise the costly process to acquire and classify images during production. This could be achieved by introducing a special lens and some generative Artificial Intelligence (AI) models. This paper proposes two novel architectures: SCWGAN-GP, Scalable Condition Wasserstein Generative Adversarial Network (WGAN) with Gradient Penalty, and Focal Stable Diffusion (FSD) model, with class injection and dedicated loss function, to artificially increase the number of images to train the DL model. In addition, a K|Lens special optics was used to get multiple views of the vertical lathe inserts as a means of further increase data augmentation by hardware with a reduced number of production samples. Given an initial dataset, the classification accuracy was increased from 80.0 % up to 96.0 % using the FSD model. We also found that using as low as 100 real images, our methodology can achieve up to 93.3 % accuracy. Using only 100 original images for each insert type and wear condition results in 93.3 % accuracy and up to 94.6 % if 200 images are used. This accuracy is considered to be within human inspector uncertainty for this use-case. Fine-tuning the FSD model, with nearly 1 billion training parameters, showed superior performance compared to the SCWGAN-GP model, with only 80 million parameters, besides of requiring less training samples to produced higher quality output images. Furthermore, the visualization of the output activation mapping confirms that the model takes a decision on the right image features. Time to create the dataset was reduced from 3 months to 2 days using generative AI. So our approach enables to create industrial dataset with minimum effort and significant time speed-up compared with the conventional approach of acquiring a large number of images that DL models usually requires to avoid over-fitting. Despite the good results, this methodology is only applicable to relatively simple cases, such as our inserts where the images are not complex.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"2016 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-05-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140942491","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Lightweight convolutional neural network for fast visual perception of storage location status in stereo warehouse","authors":"Liangrui Zhang, Xi Zhang, Mingzhou Liu","doi":"10.1007/s10845-024-02397-0","DOIUrl":"https://doi.org/10.1007/s10845-024-02397-0","url":null,"abstract":"<p>Accurate storage location status data is an important input for location assignment in the inbound stage. Traditional Internet of Things (IoT) identification technologies require high costs and are easily affected by warehouse environments. A lightweight convolutional neural network is proposed for perceiving storage status to achieve high stability and low cost of location availability monitoring. Based on the existing You Only Look Once (YOLOv5) algorithm, the Hough transform is used in the pre-processing to implement tilt correction on the image to improve the stability of object localization. Then the feature extraction unit CBlock is designed based on a new depthwise separable convolution in which the convolutional block attention module is embedded, focusing on both channel and spatial information. The backbone network is constructed by stacking these CBlock blocks to compress the computational cost. The improved neck network adds cross-layer information fusion to reduce the information loss caused by sampling and ensure perceptual accuracy. Moreover, the penalty metric is redefined by SIoU, which considers the vector angle of the bounding box regression and improves the convergence speed and accuracy. The experiments show that the proposed model achieves successful results for storage location status perception in stereo warehouse.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"22 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140928607","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Warpage detection in 3D printing of polymer parts: a deep learning approach","authors":"Vivek V. Bhandarkar, Ashish Kumar, Puneet Tandon","doi":"10.1007/s10845-024-02414-2","DOIUrl":"https://doi.org/10.1007/s10845-024-02414-2","url":null,"abstract":"<p>While extrusion-based Additive Manufacturing (AM) facilitates the production of intricately shaped parts especially for polymer processing with customized geometries, the process’s diverse parameters often lead to various defects that significantly impact the quality and hence the mechanical properties of the manufactured parts. One prominent defect in polymer-based AM is warping, which can significantly compromise the quality of 3D-printed parts. In this work, a deep learning (DL) approach based on convolutional neural networks (CNN) was developed to automatically detect warpage defects in 3D-printed parts, subsequently leading to quality control of the 3D-printed parts. Experiments were conducted using a customized Delta 3D printer with acrylonitrile butadiene styrene (ABS) and polylactic acid (PLA) materials, following the ASTM D638 tensile specimen geometry and employing design of experiments (DoE) methodology. The CNN dataset was generated by autonomously capturing high-quality (HQ) images at regular intervals using a Raspberry Pi (RPi) setup, storing the timestamped images on Google Drive, and categorizing them into ‘warped’ and ‘unwarped’ classes based on user-defined criteria. The novelty of this research lies in creating a setup for gathering image-based datasets and deploying a DL-based CNN for the real-time identification of warpage defects in 3D printed parts made of ABS and PLA materials, achieving an outstanding accuracy rate of 99.43%. This research furnishes engineers and manufacturers with a step to bolster quality control in polymer-based AM, offering automated defect correction through feedback control. By enhancing the reliability and efficiency of AM processes, it empowers practitioners to achieve higher standards of production.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"19 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140928618","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Variability-enhanced knowledge-based engineering (VEN) for reconfigurable molds","authors":"Zeeshan Qaiser, Kunlin Yang, Rui Chen, Shane Johnson","doi":"10.1007/s10845-024-02361-y","DOIUrl":"https://doi.org/10.1007/s10845-024-02361-y","url":null,"abstract":"<p>Mass production of high geometric variability surfaces, particularly in customized medical or ergonomic systems inherently display regions characterized by large variations in size, shape, and the spatial distribution. These high variability requirements result in low scalability, low production capacity, high complexity, and high maintenance and operational costs of manufacturing systems. Manufacturing molds need to physically emulate normal shapes with large variation while maintaining low complexity. A surface mold actuated with reconfigurable tooling (SMART) is proposed for molds with high variability capacity requirements for Custom Foot Orthoses (CFOs). The proposed <i>Variability Enhanced-KBE</i> (VEN) solution integrates a knowledge base of variations using statistical shape modeling (SSM), development of a parametric finite element (FE) model, a stepwise design optimization, and Machine Learning (ML) control. The experimentally validated FE model of the SMART system (RMSE &lt; 0.5mm) is used for design optimization and dataset generation for the ML control algorithm. The fabricated SMART system employs discrete coarse and fine size/shape adjustment in low and high variation areas respectively. The SMART system’s experimental validation confirms an accuracy range of 0.3-0.5mm (RMSE) across the population, showing a 84% improvement over the benchmark. This VEN SMART approach may improve manufacturing in various high variability freeform surface applications.</p>","PeriodicalId":16193,"journal":{"name":"Journal of Intelligent Manufacturing","volume":"132 1","pages":""},"PeriodicalIF":8.3,"publicationDate":"2024-05-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140942312","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}