Differentially-Private Text Generation via Text Preprocessing to Reduce Utility Loss

Abstract

To provide user-generated texts to third parties, various anonymization used to process the texts. Since this anonymization assume the knowledge possessed by the adversary, sensitive information may be leaked depending on the adversary’s knowledge even after this anonymization. Moreover, setting the strongest assumptions about the adversary’s knowledge leads to the degradation of the utility as the data by removing any quasi-identifiers. Therefore, instead of providing original data, a method to generate differentially-private synthetic data has been proposed. Differential privacy is more flexible than anonymization technologies because it does not require the assumption of the adversary’s knowledge. However, if a large noise is added to the gradient in text generative model to satisfy differential privacy, the utility of the synthetic text is degraded. Since differential privacy can be satisfied with a small noise in data containing duplicates, it is possible to reduce utility loss as text by creating duplicates before adding noise. In this study, we reduce the amount of noise added by creating duplicates through generalization, thereby minimizing text utility loss. By constructing a differentially-private text generation model, we can provide synthetic text and promote text utilization while protecting privacy information in the text.