Time and Energy trade-off analysis for Multi-Installment Scheduling with result retrieval strategy for Large Scale data processing

Abstract

On network-based computing systems, multi-instalment scheduling (MIS) has been shown to be an efficient method for reducing the time needed for processing large-scale divisible loads. Because the size of the output compared to the size of the input was considered to be minimal, the result backpropagation was not considered by traditional MIS techniques. Although the MIS-RR strategy incorporated the result retrieval phase along with a periodic MIS strategy, the energy effects of the strategy are yet to be studied. In this paper, we develop a comprehensive simulation framework using SimPy and use it to study the real-world time and energy effects of the MIS-RR strategy. We demonstrate the time and energy trade-offs that are required for an effective schedule. We analyse the strategy based on a variety of influencing factors such as network scalability, number of instalments, and overheads, and then try to determine the maximum number of processors to employ for a given load size and a given energy budget. This latter finding is crucial for system administrators when determining the number of resources to be deployed for a given load size and a power budget.