Vietnamese Question Answering System f rom Multilingual BERT Models to Monolingual BERT Model

Abstract

A question answering (QA) system based on natural language processing and deep learning gets more attention from AI communities. Many companies and organizations are interested in developing automated question answering systems which are being researched widely. Recently, the new model named Bidirectional Encoder Representation from Transformer (BERT) was proposed to solve the restrictions of NLP tasks. BERT achieved the best results in almost tasks that include QA tasks. In this work, we tried applying the multilingual BERT models (multilingual BERT [1], DeepPavlov multilingual BERT, multilingual BERT fine-tuned on XQuAD) and the language-specific BERT model for Vietnamese (PhoBERT). The obtained result has shown that the monolingual model outperforms the multilingual models. We also recommend multilingual BERT fine-tuned on XQuAD model as an option to build a Vietnamese QA system if the system is built from a multilingual BERT based model.