Multi-label text classification via hierarchical Transformer-CNN

Abstract

Traditional multi-label text classification methods, especially deep learning, have achieved remarkable results, but most of these methods use the word2vec technique to represent continuous text information, which fails to fully capture the semantic information of the text. To solve this problem, we built a hierarchical Transformer-CNN model and applied it in multi-label classification. Taking into account the characteristics of natural language, a hierarchical Transformer-CNN model is constructed to capture the semantic information of different levels of the text at the word and sentence levels using multi-headed self-attention mechanism, and a sentence convolutional neural network was used to extract key semantic features. For the hierarchical Transformer-CNN model we proposed, sufficient experiments have been conducted on the RCV1 and AAPD data sets to verify the model's effectiveness.