Question Answering System for low resource language using Transfer Learning

Abstract

This paper proposes a Question Answering System for Marathi language using Transfer Learning. A well performing Question Answering system leverages the word embeddings used in the system. Producing word embeddings for a language from the scratch is a drawn-out task and requires tremendous dataset and huge computing resources. Utilizing word embeddings created from a limited dataset in NLP tasks prompts average per-formance. Instead utilizing word embeddings from pre-trained models saves a lot of time, and gives great performance, since these models have more learnable parameters and are trained on huge datasets. Our framework uses Multilingual BERT model as pre-trained source model having 110M parameters which leads to effective word representation. We have fine-tuned this BERT model for QAS with the assistance of a small, custom dataset similar to SQuAD, intended for this framework. The system uses Bert-score and F1-score as its evaluation methods. It achieves F1-score of 56.7% and Bert-score of 69.08%. The system being the first of its kind in Marathi language lays the groundwork for future research.