Synthetic Image Assisted Deep Learning Framework for Detecting Defects During Composite Sheet Layup

Automation of high-performance manufacturing processes such as prepreg composite layup has been gaining a lot of interest lately. Reliable and accurate defect detection methods play a crucial role in the automation of such processes to maintain the desired quality. The composite prepreg layup process involves manipulation of sheet-like material. Traditional machine vision-based defect detection techniques are inept in detecting defects for such complex processes due to the nature of the defects. Advanced defect detection techniques enabled by deep learning are the key for such applications. However, Deep learning usually requires an enormous amount of physical images of the process which is infeasible in high-mix manufacturing applications. In this paper, we resolve the data generation problem for deep learning by presenting an approach where with a combination of finite element-based simulation and advanced graphics techniques we generate a dataset of photorealistic images of the defects. Approximately, 10000 synthetic images are generated and combined with around 1000 images of real sheets to train a ResNeSt-based deep learning model. We have also devised an efficient 2-stage methodology for training the deep learning network to detect wrinkle-like defects. With the trained model and data augmentation techniques, our method can achieve a mean Average Precision (mAP) of 0.98 on actual production data for detecting defects. The code and the entire dataset are available at: https://github.com/RROS-Lab/DeepSynthDefectDetector.