Researching a Distributed Computing Automation Platform for Big Data Processing

Abstract

This paper is devoted to the study of the effectiveness of methods for deploying cloud infrastructures to create a distributed computing automation platform for big data processing based on cloud infrastructures. Deployment schemes for Hadoop, Spark architecture based on Docker Swarm and Kubernetes clusters are presented. Also, the development of criteria for evaluating the effectiveness of the distributed computing automation platform for big data processing and an experimental study were carried out.