Few-shot Learning Named Entity Recognition of Pressure Sensor Patent Text Based on MLM

Abstract

Abstract of patent text, as an important support for intellectual property protection, is an ideal data source for technology mining. Named entity recognition of patent text can reduce the workload of patent analysis, improve work efficiency, and provide effective technical means for patent discovery, patent promotion, patent infringement and other aspects. However, the technical terms of patent texts are difficult to be mined, extracted and labeled. Therefore, this paper proposes a few-shot learning named entity recognition method to solve the problem that the named entity recognition of pressure sensor patent text lacks sufficient annotation data.This method uses MLM (Masked Language Model) pretraining method of BERT Model, selects a small part of token to mask each time, and then repeatedly trains on the same sample, finally obtains the training embedding of bidirectional fusion information on massive continuous corpus. Then the CRF layer is used to decode and finally the prediction tag sequence is obtained. Experiments on 55 patent abstracts and 34 patent abstracts in the field of pressure sensor preparation, the simulation results show that the proposed method can improve the recognition accuracy by about 10% compared with the traditional machine learning model (HMM, CRF) in the case of small samples. Compared with the deep learning model (BI-LSTM and BiLSTM+CRF), the accuracy of the model is improved by about 30%, and the accuracy of the model is 93%.