Efficient big data analysis on a single machine using apache spark and self-organizing map libraries

Abstract

Apache Spark is commonly used as a big data analytical platform on powerful computer clusters, as it primarily employ the main computer memory for the evaluation. Our attempt adds self-organizing map software libraries onto a single big data analytical stack and is efficient and fast enough even on a standard single computer. This innovative approach brings the big data analysis to researchers with limited resources. Our genuine idea was experimentally confirmed and is described here. As a case study for our method we we used the available #Brexit data and the sentiment analysis of corresponding tweets and the correlation with the stock exchange data.