The application of “transfer learning” in optical microscopy: the petrographic classification of metallic minerals

Abstract

Analysis of optical microscopic image data is crucial for the identification and characterization of mineral phases, and thus directly relevant to the subsequent methodology selections of further detailed petrological exploration. Here we present a novel application of Swin Transformer, a deep learning algorithm to classify metal mineral phases such as arsenopyrite, chalcopyrite, gold, pyrite, and stibnite, in images captured by optical microscopy. To speed up the training process and improve the generalization capabilities of the investigated model, we adopt the “transfer learning” paradigm by pretraining the algorithm using a large, general-purpose, image dataset named ImageNet-1k. Further, we compare the performances of the Swin Transformer with those of two well-established Convolutional Neural Networks (CNNs) named MobileNetv2 and ResNet50, respectively. Our results highlight a maximum accuracy of 0.92 for the Swin Transformer, outperforming the CNNs. To provide an interpretation of the trained models, we apply the so-called Class Activation Map (CAM), which points to a strong global feature extraction ability of the Swin Transformer metal mineral classifier that focuses on distinctive (e.g., colors) and microstructural (e.g., edge shapes) features. The results demonstrate that the deep learning approach can accurately extract all available attributes, which reveals the potential to assist in data exploration and provides an opportunity to carry out spatial quantization at a large scale (cm-mm). Simultaneously, boosting the learning processes with pre-trained weights can accurately capture relevant attributes in mineral classification, revealing the potential for application in mineralogy and petrology, as well as enabling its use in resource explorations.