Machine Reading Comprehension of High-Tech Industry Policies: A New Dataset and Chinese Pre-Trained Language Model

Abstract

Machine reading comprehension (MRC) is a challenging research hotspot in the field of Artificial Intelligence (AI). It can be applied to many scenarios, such as intelligent question answering, intelligent document retrieval, and so on. In this article, we focus on the machine reading comprehension of high-tech industry policy texts in China. First, we create a cloze style machine reading comprehension dataset of Chinese high-tech industrial policies. Next, we propose a new pre-training objective named multi-segment ordering discriminator, and we also use domain-specific dictionary to improve the MLM pre-training process. Finally, on our dataset, we trained a new pre-trained language model for machine reading comprehension of Chinese industrial policies. Experiment results show that our pre-trained language model surpasses existing models such as BERT and RoBERTa in the new dataset.