Minimizing big data problems using cloud computing based on Hadoop architecture

Abstract

This paper emphasizes importance and solution of big data problems through cloud computing. Knowledge embedded in big data generated by sensors, personal computers and mobile devices is compelling many companies to spend millions of dollars to solve problems of information and knowledge extraction to make intelligent decisions in time for the growth of their businesses. Google BigQuery, Rackspace Big Data Cloud, Amazon Web Services are some platforms that are providing limited solutions and infrastructures to deal with big data problems. However, our study motivates IT companies to use open source Hadoop architecture to develop cloud systems for reliable distributed computing to process their large data sets efficiently and effectively. Our main guideline is to resolve the big data through a company's own infrastructure and integrating various other big data infrastructures into their clouds. Also that, Hadoop reduce/map technique can be implemented on the clusters within and across the private and public clouds.