Asynchronous Distributed Matrix Factorization with Similar User and Item Based Regularization

Abstract

We introduce an asynchronous distributed stochastic gradient algorithm for matrix factorization based collaborative filtering. The main idea of this approach is to distribute the user-rating matrix across different machines, each having access only to a part of the information, and to asynchronously propagate the updates of the stochastic gradient optimization across the network. Each time a machine receives a parameter vector, it averages its current parameter vector with the received one, and continues its iterations from this new point. Additionally, we introduce a similarity based regularization that constrains the user and item factors to be close to the average factors of their similar users and items found on subparts of the distributed user-rating matrix. We analyze the impact of the regularization terms on MovieLens (100K, 1M, 10M) and NetFlix datasets and show that it leads to a more efficient matrix factorization in terms of Root Mean Square Error (RMSE) and Mean Absolute Error (MAE), and that the asynchronous distributed approach significantly improves in convergence time as compared to an equivalent synchronous distributed approach.