Store, schedule and switch - A new data delivery model in the big data era

Abstract

The big data era is posing unprecedented challenges on the existing network infrastructure. In today's networks, data are transferred across the network as a combination of a series of packets, delivered one by one, without considering the data in their entirety with respective service level requirements. The so called elephant data, which may be less sensitive to transfer delay, compete precious network resources with mice data, in most cases from interactive and delay sensitive applications. Consequently, the Quality of Service (QoS) of interactive applications is hard to provision, and the utility of network is low. We propose a new data transfer model to complement the existing per-packet forwarding paradigm. In the new data transfer model, a service level requirement is assigned (by the data source) to each big data transfer request. Instead of transferring these data on per-packet bases immediately upon entering the network, the network stores the data until it find necessary, or enough network resource is available for that transfer. The scheduled data delivery is realized through the use of dynamic circuit switching. We also present some preliminary simulation results of SSS networks.