Sentiment analysis based on Chinese BERT and fused deep neural networks for sentence-level Chinese e-commerce product reviews

Abstract

Driven by the rapid development of Internet, more e-commerce product reviews are available on e-commerce platforms, which can help enterprises make business decisions. Currently, bidirectional encoder representations from transformers (BERT) applied in the embedding layer contributes to achieve promising results in English text sentiment analysis (SA). This paper proposes a novel model Chinese BERT with fused deep neural networks (CBERT-FDNN), extracting richer and more accurate semantic and grammatical information in Chinese text. First, Chinese BERT with whole word masking (Chinese-BERT-wwm) is used in the embedding layer to generate dynamic sentence representation vectors. It is a Chinese pre-training model based on the whole word masking (WWM) technology, which is more effective for Chinese text contextual embedding. Second, multi-channel and multi-scale convolutional neural networks (CNN) and bidirectional long short-term memory (BiLSTM) are designed to capture further crucial features in the feature extraction layer. To obtain more comprehensive sentence attributes, these features are concatenated together. Last, the model is evaluated on 100,000 sentence-level Chinese e-commerce product reviews for sentiment binary classification. The accuracy and F1 score can achieve 94.37% and 94.34%, respectively. Compared with the baseline models, the experiments show that our proposed model has higher accuracy and better prediction performance.