Sentiment Analysis Implementing BERT-based Pre-trained Language Model for Vietnamese

Abstract

Continuous Improvement Process Model contributes effectively to the educational development of any school. Sentiment analysis of student feedback is a step in this model to find suitable solutions to enhance the performance of instructors and the quality of material facilities. However, most of the state-of-the-art sentiment classification models only focus on English, by which some disadvantages in Vietnamese researches are brought on. We study a sentiment analysis model using PhoBERT pre-trained model for Vietnamese, which is a robust optimization for Vietnamese of the prominent BERT model. We then employ alternative fine-tuning techniques to generalize the model for multi-class classification other than the binary task. Our method achieves state-of-the-art results on the UIT-VSFC dataset with an F1-score of 93.92% and an accuracy of 94.28%. This is expected to be helpful for the improvement of Vietnam's education and set the foundation for researching in Vietnamese which is the language that lacks resources.