Optimal Placement of Stream Processing Operators in the Fog

Abstract

Elastic data stream processing enables applications to query and analyze streams of real time data. This is commonly facilitated by processing the flow of the data streams using a collection of stream processing operators which are placed in the cloud. However, the cloud follows a centralized approach which is prone to high latency delay. For avoiding this delay, we leverage on the fog computing paradigm which extends the cloud to the edge of the network.In order to design a stream processing solution for the fog, we first formulate an optimization problem for the placement of stream processing operators, which is tailored to fog computing environments. Then, we build a plugin (for stream processing frameworks) which solves the optimization problem periodically in order to support the dynamic resources of the fog. We evaluate this approach by performing experiments on an OpenStack testbed. The results show that our plugin reduces the response time and the cost by 31.5% and 8.8% respectively, compared to optimizing the placement of operators only upon initialization.