Optimized Common Parameter Set Extraction Framework by Multiple Benchmarking Applications on a Big Data Platform

Abstract

The Apache Hadoop Distributed File System (HDFS) [1] is one of the prominent engines as a big data processing framework [2] with its distributed processing capabilities over a cluster that composed of multiple nodes [3]. The core technology of this open source is called map and reduce, which is accomplished by appropriately splitting a big task into each node and merging it through inter process communication.