Energy-Efficient and Accuracy-Aware DNN Inference With IoT Device-Edge Collaboration

Abstract

Due to the limited energy and computing resources of Internet of Things (IoT) devices, the collaboration of IoT devices and edge servers is considered to handle the complex deep neural network (DNN) inference tasks. However, the heterogeneity of IoT devices and the various accuracy requirements of inference tasks make it difficult to deploy all the DNN models in edge servers. Moreover, a large-scale data transmission is engaged in collaborative inference, resulting in an increased demand on spectrum resource and energy consumption. To address these issues, in this paper, we first design an accuracy-aware multi-branch DNN inference model and quantify the relationship between branch selection and inference accuracy. Then, based on the multi-branch DNN model, we aim to minimize the energy consumption of devices by jointly optimizing the selection of DNN branches and partition layers, as well as the computing and communication resources allocation. The proposed problem is a mixed-integer nonlinear programming problem. We propose a hierarchical approach to decompose the problem, and then solve it with a proportional integral derivative based searching algorithm. Experimental results demonstrate our proposed scheme has better inference performance and can reduce the total energy consumption up to 65.3$\%$, compared to other collaboration schemes.