Hadoop Performance Evaluation in Software Defined Data Center Networks

Abstract

The wide spread of Big Data applications and services raised the need of implementing dedicated frameworks for the efficient management of data storage and access. Complex data center architectures have been defined to support such applications, where the network connections are often the bottleneck for data access and retrieval. In this paper we analyze a data center architecture based on a Fat-Tree topology running Hadoop as the framework for data management. The network is based on the software defined networking paradigm allowing the routing protocol to be switched between the spanning tree, the shortest path bridging and the equal cost multipath. The results show that performance of Hadoop are strongly influenced by the choice of the network protocols in dependence of the traffic load. The performance evaluation is based on a dedicated novel simulation framework named MaxHadoop.