A fast and scalable supervised topic model using stochastic variational inference and MapReduce

Abstract

An important and widespread topic in cloud computing is text analyzing. People often use topic model which is a popular and effective technology to deal with related tasks. Among all the topic models, sLDA is acknowledged as a popular supervised topic model, which adds a response variable or category label with each document, so that the model can uncover the latent structure of a text dataset as well as retains the predictive power for supervised tasks. However, sLDA needs to process all the documents at each iteration in the training period. When the size of dataset increases to the volume that one node cannot deal with, sLDA will no longer be competitive. In this paper we propose a novel model named Mr.sLDA which extends sLDA with stochastic variational inference (SVI) and MapReduce. SVI can reduce the computational burden of sLDA and MapReduce extends the algorithm with parallelization. Mr.sLDA makes the training become more efficient and the training method can be easily implemented in a large computer cluster or cloud computing. Empirical results show that our approach has an efficient training process, and similar accuracy with sLDA.