Experiments with computing similarity coefficient over big data

Abstract

Big data is a hot topic nowadays due to the huge amount of data resulted from various commercial processes and also due to every day data handled by social networks. The MapReduce programming model focuses on processing and generating large data sets. Using the values obtained by computing the Jaccard similarity coefficients for two very large graphs, we have analysed the connections and influences that some nodes have over the other nodes. Furthermore, we have shown how Apache Hadoop framework and MapReduce programming model can be used for high volume computations. All tests were performed on a distributed cluster in order to obtain the results described in the paper.