A Real-time Super-Resolution for Surveillance Thermal Cameras using optimized pipeline on Embedded Edge Device

Abstract

The avenue of deep learning is scarcely explored in the domain of thermal imaging. Recovering a high-resolution output from images and videos is a classical problem in many computer vision applications. In this paper, we propose an optimized pipeline for a real-time video super-resolution task using thermal camera on embedded edge device. To tackle the challenges, we make contributions in the following several aspects: 1) comparative study of selected deep learning super-resolution models; 2) constructing and optimizing an end-to-end inference pipeline; 3) using cutting edge technology to integrate the whole workflow; 4) a real-time performance was achieved using less data; 5) we have also experimented the entire pipeline on our custom thermal dataset. As a consequence, the chosen model was able to achieve a real-time speed of over 29, 36 and 45 high FPS; 32.9dB/0.889, 31.86dB/0.801 and 30.94dB/0.728 PSNR/SSIM values for 2x, 3x and 4x scaling factors respectively.