An Architectural Design of a Job Management System Leveraging Software Defined Network

Abstract

Whether to be able to efficiently utilize the interconnect in a cluster system is a key factor in deciding the computational performance especially for a class of jobs that require intensive communications between processes. However, most of Job Management Systems (JMSs), which are deployed on cluster systems for load-balancing, do not have any mechanism that takes the pattern and requirements of communication occurred in a job into consideration, regardless of potential large impact on its performance. In this research, we explore a novel JMS that can assign computing resources to jobs submitted to the JMS from a standpoint of efficient use of both network and processor resource. More technically, in this paper, a JMS that integrates the network programming functionality of OpenFlow as Software Defined Network is proposed and discussed towards better and more efficient allocation of computing and network resources.