Attention-based Convolutional Neural Network for Answer Selection using BERT

Abstract

Question answering is at the heart of natural language processing and is composed of two sections: Reading Comprehension and Answer Selection. Prior to deep learning, all natural language processing solutions including Question Answering were based on statistical methods and researchers generated set of features based on text input. Answer Selection is a fundamental task in Question Answering, also a tough one because of the complicated semantic relations between questions and answers. Attention is a mechanism that has revolutionized deep learning community. Leveraging pretrained language models have made a breakthrough in most natural language processing tasks. Bert is one of the top pretrained deep language models that has achieved state-of-the-art on an extensive area of nlp tasks. In this paper we utilize an attention-based convolutional neural network. First, we employ BERT, a state-of-the-art pre-trained contextual as the embedding layer. Second, we enhance the model by adding some more attentive features. We evaluate the performance of our model on WikiQA dataset. Our experiments show that our model is superior to many other answer-selection models.