Improved Deep Bi-directional Transformer Keyword Extraction based on Semantic Understanding of News

Abstract

To address the problems of existing methods such as neglecting semantic information and the lack of diversity in extracted keywords. This paper proposes an improved deep bi-directional transformer model based on semantic understanding of news, combining pre-trained word vectors with K-Means algorithm. After extracting word vectors with rich semantic information based on contextual words through the bert pre-training model, then the K-Means clustering algorithm is used to form clusters of different topics. The extracted keywords semantically highlight the central theme and at the same time can better solve the problem that lack of the diversity of keywords. Experiments show that the improved deep bi-directional transformer model based on news language understanding proposed in this paper has significantly improved in accuracy, recall and F-value compared with RAKE, TF-IDF, LDA, RNN, LSTM models for extracting keywords and word2vec models for static extraction of word vectors.