VNLP: an open source framework for Vietnamese natural language processing

Abstract

Natural Language Processing (NLP) for Vietnamese has been researched for more than a decade but still lacks of an open-source NLP pipeline. As the result, researchers have to spend a lot of time on various fundamental tasks before working on the task of interest. Besides, the circumstance holds back text processing technology in Vietnam because an application costs much more money and time to reach a deliverable state. This work is an attempt to solve this issue. By incorporating available open-source software packages and implementing new ones, we have created an open-source, production-ready solution for Vietnamese text processing. Via three experiments, we demonstrated its effectiveness and efficiency. The software has helped us to develop our solution for Vietnamese sentiment analysis and online reputation management and we hope that it will also facilitate research in Vietnamese NLP.