FAS: A Flow Aware Scaling Mechanism for Stream Processing Platform Service based on LMS

Abstract

Stream Processing Platform Service (SPPS is a service built on container cloud and implemented for purpose to develop a stream processing application with simple configuration. The service needs to provide scaling ability in order to adjust system capacity for dynamic incoming data volume. Data flow is a significant indicator for system load thus it becomes a fundamental factor for analyzing. Data flow prediction thus becomes important in order to improve Quality of Service (QoS) as well as optimize resource usage. In this paper, an approach applying Least Mean Squares (LMS) on data flow prediction with a scaling mechanism for system scaling is proposed. The algorithm takes period time and data flow into consideration to predicate the required resource for processing. After the data flow prediction is calculated, decision for new coming data is made, the service scales the processing cluster in advance for predicted volume. The experiment shows the method is effective for periodically changed data flow.