Adaptive heuristics for scheduling DNN inferencing on edge and cloud for personalized UAV fleets

Abstract

Drone fleets with onboard cameras coupled with computer vision and DNN inferencing models can support diverse applications, from package deliveries to disaster monitoring. One such novel domain is for one or more “buddy” drones to assist Visually Impaired People (VIPs) lead an active lifestyle. Video inferencing tasks from such drones can help both navigate the drone and provide situation awareness to the VIP, and hence have strict execution deadlines. These tasks can execute either on an accelerated edge like Nvidia Jetson linked to the drone, or on a cloud INFerencing-as-a-Service (INFaaS). However, making this decision is a challenge given the latency and cost trade-offs across a stream of deadline-sensitive tasks, in the presence of network and/or compute variability. We propose a deadline-driven heuristic, DEMS-A, to schedule diverse DNN tasks generated continuously to perform inferencing over video segments generated by multiple drones linked to an edge, with the option to execute on the cloud. We use strategies like task dropping, work stealing and migration, and dynamic adaptation to cloud variability, to fully utilize the captive edge with intelligent offloading to the cloud, and guarantee a Quality of Service (QoS), i.e. maximize the utility and the number of tasks completed. We also introduce an additional Quality of Experience (QoE) metric useful to the assistive drone domain, which values the frequency of success for task types to ensure the responsiveness and reliability of the VIP application. We extend our DEMS solution to GEMS to solve this. We evaluate these strategies, using (i) an emulated setup of a fleet of over 80 drones supporting over 25 VIPs, with real DNN models executing on pre-recorded drone video streams, using Jetson Nano edges and AWS Lambda cloud functions, and (ii) a real-world setup of a Tello drone and a Jetson Orin Nano edge accelerator executing a subset of the DNN models on live video feeds and generating drone commands to follow a VIP in real-time. The detailed comparative emulation study shows that our strategies have a task completion rate of up to 88%, up to $2.7\times$ higher QoS utility compared to the baselines, a further 16% higher QoS utility while adapting to network variability, and up to 75% higher QoE utility. Our practical validation using real drones exhibits task completion of up to 87% for GEMS and 33% higher total utility of GEMS compared to edge-only and achieves the smoothest trajectory with minimum jerk and lowest yaw error.