Big data parallelism: Challenges in different computational paradigms

Abstract

Developers are engaged themselves in processing big data for different computational environments especially in different information systems, biological expression preparations and visual and graphical modelling. Digital Elevation Models (DEMs) in Geographic Information Systems (GIS) is one such information systems where in memory computation faces a lot challenges to manipulate and visualize the data. Scalable distributed framework broadly exhibit two design characteristics: (i) they are using memory scalability in such a manner that the amount of memory required by each process decreases as the number of processes used to solve a given problem instance increases, and (ii) they exploit coarse grain parallelism in the sense that they structure their computations into a sequence of local computation followed by communication phases in which the local computations take a non-trivial amount of time and often involve a non-trivial subset of the process' memory. In this paper we will discuss about big data, data science, different models available in the parallel paradigms, the pros and cons and the probable way out to work with high dimensional data.