Towards Transformer-Based Real-Time Object Detection at the Edge: A Benchmarking Study

Abstract

Recent work has demonstrated the success of end-to-end transformer-based object detection models. These models achieve predictive performance that is competitive with current state-of-the-art detection model frameworks without many of the hand-crafted components needed by previous models (such as non-maximal suppression and anchor boxes). In this paper, we provide the first benchmarking study of transformer-based detection models focused on real-time and edge deployment. We show that transformer-based detection model architectures can achieve 30FPS detection rates on NVIDIA Jetson edge hardware and exceed 40FPS on desktop hardware. However, we observe that achieving these latency levels within the design space that we specify results in a drop in predictive performance, particularly on smaller objects. We conclude by discussing potential next steps for improving the edge and IoT deployment performance of this interesting new class of models.