Capitalization Feature and Learning Rate for Improving NER Based on RNN BiLSTM-CRF

Abstract

Entity extraction in the natural language processing research field is still a widely researched topic. It can be a data source for the next NLP stage, such as text summarization, sentiment analysis, chatbot, machine translation, information retrieval, opinion mining, speech recognition, etc. Named Entity Recognition (NER) is the task of detecting named entities on the corpus. The detection process of entities can use various features, one of which is capital letters. Capital letters that appear at the beginning of a sentence indicate the name of a person, place, organization, geolocation, etc. The experiment uses the deep learning approach with Recurrent Neural Network Bidirectional Long Short Term Conditional Random Field (RNN-BiLSTM-CRF). Our comparing three optimization algorithms: Stochastic Gradient Descent (SGD), Adaptive Moment Estimation (Adam), and Adadelta, with the CoNLL2003 dataset. The experiment results using capital letter features showed an increase in the value of F1-Score by 2.9 higher compared to test results that did not use capital letter features. The highest F1-score score was 92.82 in testing using Adam's algorithm, with a 0.001 learning rate.