Parallel naïve Bayes regression model-based collaborative filtering recommendation algorithm and its realisation on Hadoop for big data

Abstract

Collaborative filtering (CF) algorithms are widely used in a lot of recommender systems. However, space-time overhead and high computational complexity hinder their use in large-scale systems. This paper implements the parallel naive Bayes regression model based collaborative filtering recommendation algorithm on Hadoop computing platform to scalability problem of CF. Firstly, this paper analysis the inherent parallelism of the naive Bayesian regression model and constructs the theoretical model of naive Bayesian parallelisation. Secondly, the parallel naive Bayes regression model-based collaborative filtering recommendation algorithm is realised on Hadoop platform with distributed Hadoop distributed file system (HDFS) and MapReduce as the transparent distributed infrastructure. And its temporal-spatial overhead, speedup is discussed. Finally, applying parallel the naive Bayes regression model-based collaborative filtering recommendation algorithm to a large dataset. The experiment results on Netflix dataset show that this method has high scalability and less space-time overhead, which is suitable for real-time recommendation on large dataset.