A review of named entity recognition: from learning methods to modelling paradigms and tasks

Abstract

Named Entity Recognition (NER) is commonly used when summarising news articles and legal documents. It can extract the names of politicians or organisations and help determine the aspect of a positive or negative sentiment. Previous surveys have only provided a shallow review of NER with respect to a certain datatype. In contrast, here a much deeper coverage of different approaches is provided. First articles with respect to the learning method are discussed, such as supervised or unsupervised. Next, popular models that combine two or more learning methods are introduced in a bottom-up approach. The most popular NER algorithms are compared on a recently crawled 2024 election dataset from Australia. The effect of different parameters such as number of epochs and learning rate is explored. It is concluded that pre-trained NER models are limited in their ability to model new entities and disambiguate their context. Using the sentiment score together with a state space model over entities in a sentence might help overcome these challenges.