UCL: Unit Competition of Layers for Streaming Tasks in Heterogeneous Networks

Abstract

Partitioning and offloading the deep neural network (DNN) model over multi-tier computing units have been recently proposed to shorten the inference time. However, the state-of-the-art cannot adapt to large-scale offloading problems for streaming tasks because of its exponential complexity. Besides, as an essential kind of DNNs, the offloading of grouped con-volutional neural networks (GCNNs) has not been explored yet. Motivated by the above facts, in this paper, we concentrate on the offloading of chained DNNs (CDNNs) and GCNNs for streaming tasks. Consider a typical heterogeneous network consisting of various computing units, the user equipment (UE) publishes computation-intensive and delay-sensitive streaming DNN tasks while computing units accomplish them collaboratively. To mini-mize the delay of processing the task stream, DNN layers should be offloaded to appropriate units, which is the streaming-task multi-unit (STMU) problem. To tackle this problem, we formulate a non-cooperative potential game called unit competition of layers (UCL). The theoretical analysis proves the existence of the Nash equilibrium (NE), and the corresponding algorithm with linear complexity is developed to achieve the NE. Finally, extensive experiments demonstrate that UCL outperforms the state-of-the-art significantly in large-scale scenarios while maintaining similar performance on small-scale tasks.