Dynamic DNN model selection and inference off loading for video analytics with edge-cloud collaboration

Abstract

The edge-cloud collaboration architecture can support Deep Neural Network-based (DNN) video analytics with low inference delays and high accuracy. However, the video analytics pipelines with edge-cloud collaboration are complex, involving the decision-making for many coupled control knobs. We propose a deep reinforcement learning-based approach, named ModelIO, for dynamic DNN Model selection and Inference Offloading for video analytics with edge-cloud collaboration. We jointly consider the decision-making for video pre-processing, DNN model selection, local inference, and offloading in a video analytics system to maximize performances. Our method can learn the optimal control policy for video analytics with the edge-cloud collaboration without complex system modeling. We implement a real-world testbed to conduct the experiments to evaluate the performances of our method. The results show that our method can significantly improve the system processing capacity, reduce average inference delays, and maximize overall rewards.