Knowledge Base Enhanced Topic Modeling

Abstract

Topic models, such as Latent Dirichlet Allocation (LDA), are successful in learning hidden topics and has been widely applied in text mining. There are many recently developed augmented topic modeling methods to utilize metadata information. However, the effect of topic models is still not comparable to humans. We think one key point is that humans have background knowledge, which is essential for topic understanding. Inspired by this, we propose a knowledge base enhanced topic model in this paper. We take knowledge bases as good presentations of human knowledge, with huge collections of entities and their relations. We assume that documents with related entities tend to have similar topic distributions. Based on this assumption, we compute document similarity information via the linked entities and then use it as a constraint for LDA. More specifically, we embed entities in a low-dimensional space via DeepWalk and use Entity Movers Distance to efficiently and effectively measure the similarities between documents. The results of experiments over two real-world datasets show that our method boosts the LDA model on the document classification while no supervision information is needed.