Machine Reading Comprehension Based on SpanBERT and Dynamic Convolutional Attention

Abstract

Machine reading comprehension is a challenging task in the field of natural language processing. In this paper, we propose a new neural network structure, fused SpanBERT and Dynamic convolutional Attention Network (SDANet), for span-extracted question answering, aiming to better answer questions in a given text. the main contributions and originality of SDANet are as follows: 1) using a pre-trained language model–SpanBERT to obtain a sequential representation of the text. 2) Combining dynamic convolution with a self-attentive mechanism for capturing the local and global structure of the text during text feature interaction, with a residual mechanism to enrich the sequential information. Experimental validation on the Stanford datasets (SQuAD1.1 and SQuAD2.0) was conducted that our model made progress in span-extracted reading comprehension.