A Deep Learning Method with CRF for Instance Segmentation of Metal-Organic Frameworks in Scanning Electron Microscopy Images

Abstract

This paper proposes an integrated method for recognizing special crystals, called metal-organic frameworks (MOF), in scanning electron microscopy images (SEM). The proposed approach combines two deep learning networks and a dense conditional random field (CRF) to perform image segmentation. A modified Unet-like convolutional neural network (CNN), incorporating dilatation techniques using atrous convolution, is designed to segment cluttered objects in the SEM image. The dense CRF is tailored to enhance object boundaries and recover small objects. The unary energy of the CRF is obtained from the CNN. And the pairwise energy is estimated using mean field approximation. The resulting segmented regions are fed to a fully connected CNN that performs instance recognition. The method has been trained on a dataset of 500 images with 3200 objects from 3 classes. Testing achieves an overall accuracy of 95.7% MOF recognition. The proposed method opens up the possibility for developing automated chemical process monitoring that allows researchers to optimize conditions of MOF synthesis.