Optimising Deep Learning Split Deployment for IoT Edge Networks

The Internet of Things (IoT) often generates large volumes of messy data which are difficult to process efficiently. While deep learning models have demonstrated their suitability in processing this data, the memory and processing requirements makes it difficult to deploy on edge nodes while achieving viable throughput results. Current solutions involve deploying the model in the cloud, but this leads to increased network costs due to the transfer of raw data. However, the layer based design of deep learning models allows for a model to be split into sub-models and deployed separately across IoT nodes. By deploying parts of the model on the edge node and in the cloud, the edge node is able to transmit an intermediate layer's feature output to the following sub-model instead of the raw input data. This reduces the size of the data being transmitted and results in a lower cost to the network. However, selecting the best layer to split the model becomes a multi-objective optimisation problem. In this paper, we propose an optimisation method that considers the network cost, input rate and processing overhead in selecting the best layer for splitting a model across an IoT network. We profile several popular model architectures to highlight their performance using this split deployment. Results from simulated and physical tests of the optimal layers are provided to demonstrate the method's effectiveness in real-world applications.