Automatic inventory of archaeological artifacts based on object detection and classification using deep and transfer learning

Abstract

The inventory of a large collection of archaeological artifacts can be a tedious and time-consuming task. However, nowadays it is possible to reduce its complexity through the use of artificial intelligence tools, including object detection and classification. Deep learning is particularly an AI method which is highly effective for information retrieval and exploration of big datasets. In this work, a technique based on deep learning is applied on an archaeological dataset documenting the discoveries made by the Italian archaeological team at the Al-Baleed site in Oman. The suggested method seeks to: (a) segment photos into individual artifacts; (b) define the segmented artifacts (e.g., pottery, vessel pieces, jewellery, etc.); and (c) categorize the recognized items based on the material used in their handcraft (e.g., earthenware, glass, metal alloy, etc.). Two different kinds of deep neural network models were used to accomplish this twofold function. The first one was used for object identification and was based on Google’s TensorFlow2 Object Detection API, while the second one was created from scratch and trained to categorize the materials of an artifact. An on-site photo collection served as the dataset for training, validating, and testing both varieties of neural nets. However, data augmentation was carried out to provide more training sample versions in order to improve the models’ generalization ability. Evaluation was achieved using standard metrics for each task, such as the mean Average Precision (mAP) for object identification and the overall Accuracy for classification. The findings indicate a good rate of object detection and identification and, more importantly, a satisfactory accuracy of the artifact material’s classification. Besides, benchmarking with state-of-the-art image classification methods, based on transfer learning models, namely SqueezeNet and GoogleNet, which are trained on bigger datasets such as ImageNet, show that the accuracy of the proposed approach attain a comparable accuracy, with the advantage to be specifically trained on the studied dataset. As a result, the models could potentially be used to firstly for creating an automatic inventory process for the archaeological artifacts, and secondly uncover patterns in archaeological data that are currently unknown to assist the identification of items within a sizeable dataset.