Research on the Computing Framework in Big Data Environment

Abstract

Computing framework is one of the key technologies in improving data analytics and processing efficiency. Since open source big data computing platform Hadoop was born ten years ago, many research achievements have been made in information acquisition, analytical processing and integrated services. Several improved frameworks were proposed against the limitations of the first generation of Map Reduce version 1 (MRv1) in scalability, reliability, efficient utilization of resource, and multiple computing model supports. This paper presents and analyzes these research results, such as batch computing framework, iterative computing framework, interactive computing framework, stream computing framework, and real-time computing framework. Undoubtedly, more targeted computing models will be generated in different application fields in the future, and these computing frameworks will play an increasingly important role in the field of big data.