Nighttime Vehicle Classification based on Thermal Images

Each Department of Transportation in the United States must provide to the Federal Highway Administration on annual basis the number and types of vehicles traveled on its state-maintained roads. These data are fed into the Highway Performance Monitoring System used to assess the nation’s highway system performance. Classifying vehicles (i.e., identifying their types, e.g., passenger cars, trucks, etc.) during nighttime is quite challenging due to limited lighting. This study designed and evaluated three Convolutional Neural Network (CNN) models to classify vehicles using their thermal images. These three models have architectures that differ in the number of layers and, in the case of the third model, the addition of an inception layer. Of these, the second model achieves the best performance, achieving mean accuracy scores of greater than 97% for each of the three vehicle classes and f1 scores of greater than 98%. We proposed two training-test methods based on data augmentation to avoid over-fitting and to improve performance. The experimental results demonstrated that a data augmentation training-test method improves model performance further with regard to both accuracy and f1-score.