A measurement-based calibration approach for highly scalable timing and energy modeling of EdgeAI multi-core systems

Abstract

Deploying Artificial Neural Networks (ANNs) on embedded multi-core platforms requires precise models for estimating and optimizing timing and energy, which is crucial for enabling novel Artificial Intelligence (AI) applications. However, predicting non-functional properties (timing, power) is challenging due to degrees of parallelism in ANNs and complex effects in execution platforms (e.g. contentions at shared resources, dynamic power management). This article presents an Electronic System-Level (ESL) timing and energy modeling flow and the associated calibration methodology for optimizing ANN deployment on multi-core platforms. The proposed flow leverages SystemC simulation to offer both speed and accuracy while ensuring high scalability in many dimensions, such as platform resources modeling. Analytical models are used for ANN layer computation and communication delays as well as power consumption and energy cost. We propose a measurement-based calibration approach to these models which enables high prediction accuracy while guaranteeing high re-usability. The calibrated models can be used across different settings without the need to re-perform a calibration phase. We validate our flow against real measurements of ANN implementations on a prototype multi-core platform. Results demonstrate over 97% accuracy in timing and 93% in energy for 54 mappings of different ANNs tested with and without the use of power management on the platform, with an evaluation time under 2s per mapping. Furthermore, we illustrate that our flow is suitable for Design Space Exploration (DSE), allowing up to 24% improvement in inference time and 16% in energy compared to baseline implementation.