Research on Application of Named Entity Recognition of Electronic Medical Records Based on BERT-IDCNN-CRF Model

Abstract

Bi-LSTM-CRF (Bi-Directional Long Short-Term Memory Conditional Random Field) model have good performance in Chinese medical Electronic Medical Records (EMRS) Named Entity Recognition (NER), However, Bi-LSTM-CRF model cannot make full use of the parallelism of GPU (Graphics Processing Unit) in massive medical records, and the neglect of word order features and semantic information in IDCNN(Iterated Dilated Convolutional Neural Networks) model leads to poor NER effect. Therefore, this paper proposes a BERT-IDCNN-CRF model. In this model, the two-way transformer pre training model BERT is used to fine tune the model parameters in the manual annotated corpus conforming to the BIOES (Begin Inside Outside End Single) standard. The text is learned in an unsupervised manner, and the semantic information of words is represented by word vectors, which can well represent the context semantics in the sentences of EMRS; The state characteristics of character sequences are learned through BERT model, and the sequence state scores obtained are input to the CRF layer. The CRF layer makes constraint optimization on the sequence state transition, and IDCNN has better recognition effect on convolutional coding of local entities. Experimental test results: the average accuracy, recall and F1 value of the BERT-IDCNN-CRF model are 94.5%, 93.8% and 94.1% respectively, which are increased by 4.8%, 4.3% and 3.6% respectively compared with the baseline model Word2Vec-BiLSTM-CRF. The experiment proves that the BERT-IDCNN-CRF model can better identify medical entities in electronic medical records.