ODLIE: On-Demand Deep Learning Framework for Edge Intelligence in Industrial Internet of Things

Recently, we have witnessed the evolution of Edge Computing (EC) and Deep Learning (DL) serving Industrial Internet of Things (IIoT) applications, in which executing DL models is shifted from cloud servers to edge devices to reduce latency. However, achieving low latency for IoT applications is still a critical challenge because of the massive time consumption to deploy and operate complex DL models on constrained edge devices. In addition, the heterogeneity of IoT data and device types raises edge-cloud collaboration issues. To address these challenges, in this paper, we first introduce ODLIE, an on-demand deep learning framework for IoT edge devices. ODLIE employs DL right-selecting and DL right-sharing features to reduce inference time while maintaining high accuracy and edge collaboration. In detail, DL right-selecting chooses the appropriate DL model adapting to various deployment contexts and user-desired qualities, while DL right-sharing exploits W3C semantic descriptions to mitigate the heterogeneity in IoT data and devices. To prove the applicability of our proposal, we present and analyze latency requirements of IIoT applications that are thoroughly satisfied by ODLIE.