Load balancing scheme for supporting real-time processing of big data in distributed in-memory systems

Abstract

In this paper, we propose a new load balancing scheme which performs data migration or replication according to the loading conditions in heterogeneous distributed in-memory environments. The proposed scheme replicates hot data when the hot data occurs on the node where a load occurs. If the loading of the node increases in the absence of hot data, the data is migrated through an adjustment of the storage space. It was shown through various performance evaluations that the proposed load balancing scheme improved the overall load balancing performance.