Testing MapReduce program using Induction Method

Abstract

MapReduce is "divide and conquer" applied paradigm for processing large volume of data to filter out information to solve day to day complex challenges. MapReduce is core of big data applications. The challenging part to test these applications which also represent the characteristic of these applications are variation in data due to different format and sources. In other words, poor quality of input data can deviate system towards failure if not handled properly programmatically for variety of input data. MapReduce program itself based on transformations at different level based on the program logic This paper proposes the testing technique based on the mathematical induction principle and considered as extension or conjunction other testing techniques already in used either based on transformations analysis from input to output as in MRFlow. Proposed function testing can be used in business acceptance testing and showcase the correctness of program, further can detect many defects even before shipping bigdata application in live.