An improved algorithm of TFIDF combined with Naive Bayes

Abstract

The TF-IDF algorithm is often used for the extraction of keywords of articles, but it only considers the information of word frequency, which limits the choice of keywords. In order to improve the efficiency of the algorithm, an improved algorithm has been presented, which adds the synonyms of keywords trained by word2vec model to the word vector composed of keywords. Then the improved algorithm is given different weights based on the part of speech and the location information. And combine the improved algorithm with the Naive Bayes algorithm. In order to verify the effectiveness of the improved algorithm, experiments were conducted on a standard data set. The experimental results show that, compared with the traditional method, the accuracy of the improved TFIDF algorithm combined with Naive Bayes is greatly improved.