CNN Performance Prediction on a CPU-based Edge Platform

Abstract

The implementation of algorithms based on Deep Learning at edge visual systems is currently a challenge. In addition to accuracy, the network architecture also has an impact on inference performance in terms of throughput and power consumption. This demo showcases per-layer inference performance of various convolutional neural networks running at a low-cost edge platform. Furthermore, an empirical model is applied to predict processing time and power consumption prior to actually running the networks. A comparison between the prediction from our model and the actual inference performance is displayed in real time.