Faster R-CNN based Traffic Sign Detection and Classification

Abstract

Traffic sign is the key aspect in road and also for the autonomous car. Detection and classification of these sign plays a vital role for the invention of driverless vehicles. Convolutional neural network (CNN) has the ability to learn local features using series of convolutional and pooling layer observing the image sequences. In this work, traffic sign detection and classification has been performed based on deep learning approach. The experiment conducted on Germen Traffic Sign Detection Benchmark (GTSDB) and Recognition Benchmark (GTSRB) for detection and recognition. For traffic sign detection a two-stage detector, Faster R-CNN with ResNet 50 backbone structure is used where the CNN layers extracted the features of traffic signs from the images and the region proposal network (RPN) filter the object from the image to create bounding box based on the extracted feature map. The classification network classifies the traffic signs and predict the proposal confidence score. A general deep learning model is transferred into a specific output with weights with transfer learning by tuning the pretrained model based on COCO image dataset. The performance is compared with ResNet 152, MobileNet v3 and RetinaNet based on the confidence score and mean average precision (mAP). Faster R-CNN with ResNet-50 shows better detection performance comparing with other backbone structure. In addition, a series of convolution layer with batch normalization followed by max pooling layer is used to build a classifier and softmax is used in the output for 43 class classification and 97.89% test accuracy has been obtained.