A deep learning-based approach to extraction of filler morphology in SEM images with the application of automated quality inspection

Abstract Automatic extraction of filler morphology (size, orientation, and spatial distribution) in Scanning Electron Microscopic (SEM) images is essential in many applications such as automatic quality inspection in composite manufacturing. Extraction of filler morphology greatly depends on accurate segmentation of fillers (fibers and particles), which is a challenging task due to the overlap of fibers and particles and their obscure presence in SEM images. Convolution Neural Networks (CNNs) have been shown to be very effective at object recognition in digital images. This paper proposes an automatic filler detection system in SEM images, utilizing a Mask Region-based CNN architecture. The proposed system can simultaneously classify, detect, and segment fillers in SEM images, making it suitable for morphology analysis of fillers and automatic quality inspection. We also propose a novel SEM image simulation procedure to overcome the data scarcity for training a deep CNN architecture. The proposed filler detection system is trained on the simulated images. It is shown that the trained network can detect and segment fillers with higher accuracy even in the overlapping and obscure situations. The performance and robustness of the proposed system are evaluated using both simulated and real microscopic images.