EFFECT-DNN: Energy-efficient Edge Framework for Real-time DNN Inference

Abstract

Real-time visual computing applications running Deep Neural Networks (DNN) are becoming popular for mission-critical use cases such as, disaster response, tactical scenarios, and medical triage that require establishing ad-hoc edge environments. However, strict latency deadlines of such applications require real-time processing of pre-trained DNN layers (i.e., DNN inference) involving image/video data which is highly challenging to achieve under such resource- constrained edge environments. In this paper, we address the trade-off between end-to-end latency of DNN inference and IoT devices’ energy consumption by proposing ‘EFFECT-DNN’, an energy efficient edge computing framework. The EFFECT-DNN framework aims to strike such balance by employing a collaborative DNN partitioning and task offloading strategy. Such strategy also involves resource allocation from IoT devices and edge servers to satisfy DNN inference deadline requirement even when the network bandwidth is on the lower end, which is often the case for critical use cases. The underlying optimization is formulated as a dynamic Mixed-Integer Nonlinear Programming (MINLP) problem is decoupled and solved by convex optimization and a game-like heuristic algorithm. We evaluate the performance of EFFECT-DNN framework on a hardware testbed and using extensive simulations with real-world DNN s. The results demonstrate that the proposed framework can ensure DNN inference deadline satisfaction with significant (~ 20-30%) device energy savings.