TinderAI: Support System for Matching AI Algorithms and Embedded Devices

Abstract

Artificial Intelligence (AI) is becoming increasingly important and pervasive in the modern world. The widespread adoption of AI algorithms is reflected in the extensive range of HW devices on which they can be deployed, from high-performance computing nodes to low-power embedded devices. Given the large set of heterogeneous resources where AI algorithms can be deployed, finding the most suitable device and its con- figuration is challenging, even for experts. We propose a data-driven approach to assist AI adopters and developers in choosing the optimal HW resource. Our approach is based on three key elements: i) fair benchmarking of target AI algorithms on a set of hetero- geneous platforms, ii) creation of ML models to learn the behaviour of these AI algorithms, and iii) support guidelines to help identify the best deployment option for a given AI algorithm. We demonstrate our approach on a specific (and relevant) use case: Deep Neural Net- work (DNN) inference on embedded devices.