A privacy-preserving LDA model training scheme based on federated learning

Abstract

Latent Dirichlet Allocation (LDA) is a widely used topic modeling technique that effectively extracts the distribution of topics and their associated words from various types of textual data. However, during the iterative training of an LDA model, there is a risk of leaking sensitive text information. Additionally, many current LDA training methods rely on centralized training patterns, which pose several challenges. In this manner, it can be difficult for the training node to process large volumes of text simultaneously. This setup also makes the node a single point of failure, a potential performance bottleneck, and a target for attackers. For these issues, this paper introduces an adaptive distributed training framework (FedLDA), combining federated learning and Collapsed Gibbs Sampling (CGS) for distributed datasets. Furthermore, we present a privacy-preserving LDA model training scheme (FedLDA-DP) that combines FedLDA and differential privacy technology. Analysis and experimental results demonstrate the effectiveness and efficiency of the proposed scheme.