A Novel Architecture of Scheduling System for Big Data Framework

Abstract

The calculation of massive data has always been an significant research topic, and the data calculation job scheduling system is an important platform to deal with this problem. There are many scheduling systems in use, however, their functions are not perfect and cannot meet the needs of users. This paper designs a novel architecture of scheduling system for big data framework. The system has basic scheduling functions, such as algorithm job assignment, job priority scheduling, job result management. It supports three algorithms to schedule jobs: scheduling-in-order algorithm, scheduling-by-priority algorithm, and scheduling-by-resource algorithm, meanwhile implementing dynamic switching of these algorithms. The system has a graphical interface which greatly facilitates interaction between the system and the user.