Text Infilling Method based on Key Semantic Information Selection Mechanism

Abstract

Aiming at the problem that the lack of key semantic information in text infilling leads to weak semantic coherence of the filled text, this paper designs a text infilling method based on a Bi-directional long-short term memory network with a key semantic information selection mechanism. First, this paper uses the Bi-directionallong-short term memory network to capture the characteristics of potential long-distance dependencies and obtain context hiding features; then, we design an information selection mechanism to obtain context-critical semantic features by calculating the semantic distribution weight of words in the text; and finally, through the multi-head attention mechanism the key information of the local context and the global semantic information will be captured to fill the missing parts one by one, thereby improving the logic of the semantics and making the generated text more coherent. The experimental results show that testing on the Yelp dataset, Grimm dataset and Chinese poetry dataset, the value of the semantic fluency evaluation index perplexity has all decreased, indicating that the method proposed in this paper significantly improves the semantic coherence of the text, which significantly outperform the state-of-the-art text infilling model.