Software-defined networking enabled big data tasks scheduling: A tabu search approach

Abstract

The growth of information technology along with the revolution of the industry and business has led to the generation of an enormous amount of data. This big data needs a platform beyond the traditional data possessing context that relies on some computational servers communicating through a network in its lower layer. One of the most important challenges in data processing is how to transfer the big batches of data between the servers to achieve fast responsiveness. Consequently, the underlying network plays a critical role in the performance of a big data analysis platform. Ideally, this network must use the shortest path that has the lowest amount of load, to transfer the large-scale data. To address this issue, we propose a software-defined networking (SDN) enabled scheduling method that uses the tabu search algorithm to schedule big data tasks. The proposed algorithm not only considers data locality but also uses the network traffic status for efficient scheduling. Our extensive simulative study in the Mininet emulator shows that the proposed scheme gives high performance and minimizes job completion time.