Edge-Computing-Aware Deployment of Stream Processing Tasks Based on Topology-External Information: Model, Algorithms, and a Storm-Based Prototype

Abstract

Stream Processing Frameworks (SPF, e.g., Apache Storm) are solutions that facilitate and manage the execution of processing topologies that consist of multiple parallelizable steps (or tasks) and involve continuous data exchange among these tasks. Stemming from the world of Cloud-centric Big Data processing, SPFs often fail to address certain requirements of Internet-of-Things systems. For example, existing deployment solutions ignore the fact that topology tasks can also be involved in other interactions and data-intensive communication flows, which are not taking place between the tasks, but between a task and another Internet-of-things entity, such as an actuator, a database, or a user. This paper describes SPF extensions for taking these interactions into account. The extensions are described both generically and as extensions of Apache Storm. In a simple evaluation upon a topology which involves topology-external interactions, we demonstrate how our solution can eliminate latency requirements violations and reduce Cloud-to-edge bandwidth consumption to 1/3 compared to Apache Storm.