Linear Quadratic Regulator for Resource-Efficient Cloud Services

1 PROBLEM AND MOTIVATION The run-time performance of modern applications deployed within containers in the cloud critically depends on the amount of provisioned resources. Provisioning fewer resources can result in performance degradation and costly SLA violations; while allocating more resources leads to wasted money and poor resource utilization. Moreover, these applications undergo striking variations in load patterns. To automatically adapt resources in response to changes in load, an autoscaler applies predefined heuristics to add or remove resources allocated for an application based on usage thresholds. However, it is extremely challenging to configure thresholds and scaling parameters in an applicationagnostic manner without deep workload analysis. Poor resource efficiency results in high operating costs and energy expenditures for all cloud-based enterprises. To improve resource efficiency over safe and hand-tuned autoscaling schemes reinforcement learning (RL) based approaches [3, 4] learn an optimal scaling actions through experience (trial-anderror) for every application state, based on the input workload, or other variables. After the learning agent executes an action, it receives a response from the environment, based on the usefulness of the action. The agent is inclined to execute actions that provide higher rewards, thus reinforcing better actions. However, these proposed approaches suffer from the curse of dimensionality [2]. The state and action space to be discretized grows exponentially with the number of state variables, leading to scalability problems, manifested in unacceptable execution times in updation and selection of the next action to be executed in online setting. Moreover, these methods require huge amount of of samples to learn and thus often lack stability and interpretability of policy.