Multi-level Feature Fusion Method for Long Text Classification

Abstract

News classification task is essentially long text classification in the field of NLP (Natural Language Processing). Long text contains a lot of hidden or topic-independent information. Moreover, BERT (Bidirectional Encoder Representations from Transformer) can only process the text with a character sequence length of 512 at most, which may lose the key information and reduce the classification effectiveness. To solve above problems, the paper puts forward a model of mutli-level feature fusion based on BERT, which is suitable for the BERT through the hierarchical decomposition of long text. Then CNN (Convolutional Neural Networks) and stacked BiLSTM (Bidirectional Long Short-term Memory) based on attention mechanism are used to capture local and contextual features of text respectively. Finally, various features are spliced for classification task. The experimental results show that the model achieves 97.4% accuracy and 97.2% F1 score on THUCNews, 1.2% accuracy and 1.6% F1 score higher than that of BERT-CNN, 1.8% accuracy and 1.4% F1 score higher than that of BERT-BiLSTM, indicating that our model can significantly improve the effectiveness of news classification.