Vertical Workflows: Service Orchestration across Cloud & Edge Resources

Abstract

Currently devices used for data capture often differ from those that are used to subsequently carry out analysis on such data. Many Internet of Things (IoT) applications today involve data capture from sensors that are close to the phenomenon being measured, with such data subsequently being transmitted to Cloud data centers for analysis and storage. Increasing availability of storage and processing devices closer to the data capture device, perhaps over a one-hop network connection or even directly connected to the IoT device itself, requires more efficient allocation of processing across such edge devices and data centers. We refer to these as "vertical workflows" – i.e. workflows which are enacted across resources that can vary in: (i) type and behaviour; (ii) processing and storage capacity; (iii) latency and security profiles. Understanding how a workflow pipeline can be enacted across these resource types is outlined, motivated through two scenarios. The overall objective considered is the completion of the workflow within some deadline constraint, but with flexibility on where data processing is carried out.