A short text topic modeling method based on integrating Gaussian and Logistic coding networks with pre-trained word embeddings

Abstract

The development of neural networks has provided a flexible learning framework for topic modeling. Currently, topic modeling based on neural networks has garnered wide attention. Despite its widespread application, the implementation of neural topic modeling still needs to be improved due to the complexity of short texts. Short texts usually contains only a few words and a small amount of feature information, lacking sufficient word co-occurrence and context sharing information. This results in challenges such as sparse features and poor interpretability in topic modeling. To alleviate this issue, an innovative model called Topic Modeling of Enhanced Neural Network with word Embedding (ENNETM) was proposed. Firstly, we introduced an enhanced network into the inference network part, which integrated the Gaussian and Logistic coding networks to improve the performance and the interpretability of topic extraction. Secondly, we introduced the pre-trained word embedding into the Gaussian decoding network part of the model to enrich the contextual semantic information. Comprehensive experiments were carried out on three public datasets, 20NewGroups, AG_news and TagMyNews, and the results showed that the proposed method outperformed several state-of-the-art models in topic extraction and text classification.