Adaptive DNN Partition in Edge Computing Environments

Abstract

Deep Neural Network (DNN) has been applied widely nowadays, making remarkable achievements in a wide variety of research fields. With the improvement of the accuracy requirements for the inference results, the topology of DNN tends to be more and more complex, evolving from chain topology to directed acyclic graph (DAG) topology, which leads to the huge amount of computation. For those end devices which have limited computing resources, the delay of running DNN models independently may be intolerable. As a solution, edge computing can make use of all available devices in the edge computing environments comprehensively to run DNN inference tasks, so as to achieve the purpose of acceleration. In this case, how to split DNN inference task into several small tasks and assign them to different edge devices is the central issue. This paper proposes a load-balancing algorithm to split DNN with DAG topology adaptively according to the environment. Extensive experimental results show the the propose adaptive algorithm can effectively accelerate the inference speed.