Named Entity Recognition Based on BERT-BiLSTM-SPAN in Low Resource Scenarios

Abstract

The task of named entity recognition (NER) is crucial in the creation of knowledge graphs. With the advancement of deep learning, the pre-training model BERT has become the mainstream solution for NER. However, lack of corpus leads to poor performance of NER models using BERT alone. In low resource scenarios, previous work has focused on merging complex information to model or transfer learning from high resource corpora. Therefore, a simple but effective strategy for fully utilizing the corpus is required. In this paper, we focus on recognizing entities under resource constraints. We propose BERT-BiLSTM-SPAN for low resource scenarios, where BERT is used as an embedding layer, combined with BiLSTM and a decoding layer using a span pointer decoding algorithm. To make our model more robust, we employ adversarial training and data augmentation techniques. We conduct experiments on the marine news dataset. The BERT-BiLSTM-SPAN achieves an 80.11% F1-score. Furthermore, experimental results of data augmentation and adversarial training are both encouraging. Therefore, our proposed solutions show suitability in low resource scenarios.