RCNet: Resilient Collaborative DNN Inference for Wireless Networks With High Packet Loss

Abstract

The limited computation resources of mobile devices hinders the real-time Deep Neural Network (DNN) inference, which is critical in many Internet of Things (IoT) applications. To meet the real-time responses demands, the cloud-end collaborative DNN inference is promising, which partially offloads the inference workloads from mobile devices to the cloud server with powerful computation resources through wireless networks. However, in many IoT applications, the wireless networks are of poor link conditions with high packet loss rates, which has posed a substantial obstacle to the intermediate feature transmission. In such scenarios, it is rather challenging to achieve efficient and resilient collaborative DNN inference. In this paper, we tackle this challenge by proposing a Resilient Collaborative DNN inference framework, named RCNet, to maintain high accuracy under high packet loss conditions in wireless networks. It leverages an unequal redundant encoding mechanism to efficiently prioritize the successful transmission of important features on the mobile devices, and a Transformer-based feature reconstruction module to fully leverage the powerful computation resources on the cloud server to recover the missing features. We implement a real-world testbed and conduct extensive experiments. The experimental results verify that RCNet enables robust collaborative inference with an accuracy surpassing 90%, even under extremely harsh network conditions with over 90% of features being lost.