Automatic Parameter Tuning for Big Data Pipelines with Deep Reinforcement Learning

Tuning big data frameworks is a very important task to get the best performance for a given application. However, these frameworks are rarely used individually, they generally constitute a pipeline, each having a different role. This makes tuning big data pipelines an important yet difficult task given the size of the search space. Moreover, we have to consider the interaction between these frameworks when tuning the configuration parameters of the big data pipeline. A trade-off is then required to achieve the best end-to-end performance. Machine learning based methods have shown great success in automatic tuning systems, but they rely on a large number of high quality learning examples that are rather difficult to obtain. In this context, we propose to use a deep reinforcement learning algorithm, namely Twin Delayed Deep Deterministic Policy Gradient, TD3, to tune a fraud detection big data pipeline. We show through the conducted experiments that the TD3 agent improves the overall performance of the pipeline by up to 63% with only 200 training steps, outperforming the random search on the high-dimensional search space.