AI-based Pilgrim Detection using Convolutional Neural Networks

Abstract

Pilgrimage represents the most important Islamic religious gathering in the world where millions of pilgrims visit the holy places of Makkah and Madinah to perform their rituals. The safety and security of pilgrims is the highest priority for the authorities. In Makkah, 5000 cameras are spread around the holy mosques for monitoring pilgrims, but it is almost impossible to track all events by humans considering the huge number of images collected every second. To address this issue, we propose to use an artificial intelligence technique based on deep learning and convolutional neural networks to detect and identify Pilgrims and their features. For this purpose, we built a comprehensive dataset for the detection of pilgrims and their genders. Then, we develop two convolutional neural networks based on YOLOv3 and Faster-RCNN for the detection of Pilgrims. Experiment results show that Faster RCNN with Inception v2 feature extractor provides the best mean average precision over all classes (51%). A video demonstration that illustrates a real-time pilgrim detection using our proposed model is available at [1].