Incorporating edge sharpening and covariance attention for named entity recognition

Named Entity Recognition (NER) is a key application in the field of Artificial Intelligence and Natural Language Processing, which automatically identifies and categorizes entities in text by intelligent algorithms. In NER, all spans of a sentence can be organized into a two-dimensional representation. The semantic plane has the advantage to represent the semantic structure of a sentence and to learn the interaction between spans. One of an important phenomenon of this representation is that neighboring elements of the semantic plane are spans denoted to overlapped subsequences in a sentence. Because they share the same contextual features and semantic dependencies, it is difficult to distinguish true entities from the backgrounds. Therefore, refining span representations and building the semantic dependency between spans is helpful for the entity recognition task. In this paper, we propose an Edge Sharpening and Covariance Attention (ES&CA) model to support recognizing named entities from the semantic plane representation. The edge sharpening (ES) module adopts a differential convolution to sharpen the semantic gradients in the semantic plane, which has the ability to gather semantic information from neighborhoods. In the covariance attention (CA) module, the covariance between spans are applied to weight the attention of spans relevant to task-relevant learning objective. Establishing semantic relationships across spans is a highly successful approach. The ES&CA model is assessed on five public datasets for nested and flattened named entity recognition. The evaluation results demonstrate the effectiveness of our strategy in distinguishing entity spans from the backgrounds, hence significantly enhancing the final performance.