Fine-Tuning BERT for Sentiment Analysis of Vietnamese Reviews

Abstract

Sentiment analysis is an important task in the field of Nature Language Processing (NLP), in which users' feedback data on a specific issue are evaluated and analyzed. Many deep learning models have been proposed to tackle this task, including the recently-introduced Bidirectional Encoder Representations from Transformers (BERT) model. In this paper, we experiment with two BERT fine-tuning methods for the sentiment analysis task on datasets of Vietnamese reviews: 1) a method that uses only the [CLS] token as the input for an attached feed-forward neural network, and 2) another method in which all BERT output vectors are used as the input for classification. Experimental results on two datasets show that models using BERT slightly outperform other models using GloVe and FastText. Also, regarding the datasets employed in this study, our proposed BERT fine-tuning method produces a model with better performance than the original BERT fine-tuning method.