Practical Object Detection Using Thermal Infrared Image Sensors

Abstract

Reliable object detection is critical for autonomous vehicles (AV). An AV must be safely guided towards its destination under different illumination conditions and avoid obstacles. Thermal infrared (TIR) camera sensors can provide robust image quality under any illumination. Past object detection work using TIR sensors focused on detecting only pedestrians by filtering thermal values. Other approaches leveraged the advantages of a pre-trained RGB-based model. However, the thermal threshold-based filtering can increase false positives depending on the TIR camera capability. Moreover, a large and new TIR training dataset is needed to improve the accuracy of the RGB-based object detection networks. The time and effort to annotate new data are significantly high. In this paper, we propose efficient and practical approaches to provide robust object detection from TIR images. We first reduce the cost of training with new data by using an automated process. To increase the final object detection accuracy, we next propose fusion methods that combine results from dual TIR camera sensors. Finally, we substantiate the practical feasibility of our approach and evaluate the substantial improvement in object detection accuracy. We use various detection networks and datasets on discrete Nvidia GPUs and an Nvidia Xavier embedded platform, commonly used by automotive OEMs.