Improving the Quality of Inference for Applications using Chained DNN Models during Edge Server Handover

Recent advances in deep neural networks (DNNs) have greatly benefited mobile applications that perform real-time video analytics. However, mobile devices' computing power usually limits them from inferring complex DNN models timely. Edge intelligence has emerged to help mobile apps offload DNN inference tasks to powerful edge servers for accelerated inference services. One major challenge that edge intelligence faces is maintaining a satisfactory quality of service when users move across edge servers. To address this issue, we propose a novel solution to help improve the quality of inference services for real-time video analytics applications that use chained DNN models. This solution includes two schemes: one maximizes the use of mobile devices to improve inference quality during the handover between edge servers, and the other provides offloading decisions to minimize the end-to-end inference delay when edge servers are available. We evaluate the proposed scheme using a DNN-based realtime traffic monitoring application through testbed and simulation experiments. The results show that our solution can improve inference quality by 52% during handover compared to the greedy algorithm-based solution.