Diffulex: Diffusion based lexically constrained text generation with mixed absorbing state and constraint balance

Lexically constrained text generation aims to generate complete text with given keywords, which can be applied in many fields, such as dialogue systems, automatic summarization, and story generation. However, the current methods often find it difficult to strike a balance between generation quality, constraint ability, and generation speed, and most of them can only focus on one aspect, which seriously limits their applications. To solve this problem, we propose Diffulex, a lexically constrained text generation model based on the diffusion model, which achieves faster generation speed and higher flexibility. In response to the characteristics of lexically constrained text generation tasks, Diffulex employs a forward process of mixed absorbing states, converting tokens into different types of [MASK] tags to capture the semantic relationship between constraints and tokens. Through the constraint balance of the reverse process, more attention will be paid to the prediction tokens that meet the constraint conditions and promote the dynamic fusion of the hidden state constraint information, achieving the balance between the generation quality and the constraint ability. We compared Diffulex with advanced work in the field and popular large language models as baselines, and our results on multiple datasets show that Diffulex outperforms the baseline in various aspects. Our code is available on https://github.com/Kenfree0/Diffulex.