Micro-Workflows Data Stream Processing Model for Industrial Internet of Things

Abstract

The fog computing paradigm has become prominent in stream processing for IoT systems  where cloud computing struggles from high latency challenges. It enables the deployment of computational  resources between the edge and cloud layers and helps to resolve constraints, primarily  due to the need to react in real-time to state changes, improve the locality of data storage, and  overcome external communication channels’ limitations. There is an urgent need for tools and  platforms to model, implement, manage, and monitor complex fog computing workflows. Traditional  scientific workflow management systems (SWMSs) provide modularity and flexibility to  design, execute, and monitor complex computational workflows used in smart industry applications.  However, they are mainly focused on batch execution of jobs consisting of tightly coupled  tasks. Integrating data streams into SWMSs of IoT systems is challenging. We proposed a microworkflow  model to redesign the monolith architecture of workflow systems into a set of smaller  and independent workflows that support stream processing. Micro-workflow is an independent  data stream processing service that can be deployed on different layers of the fog computing  environment. To validate the feasibility and practicability of the micro-workflow refactoring, we  provide intensive experimental analysis evaluating the interval between sensor messages, the time  interval required to create a message, between sending sensor message and receiving the message  in SWMS, including data serialization, network latency, etc. We show that the proposed decoupling  support of the independence of implementation, execution, development, maintenance, and  cross-platform deployment, where each micro-workflow becomes a standalone computational unit,  is a suitable mechanism for IoT stream processing.