Handling out of vocabulary words at the semantical level using recurrent neural networks

Abstract

Text recognition through natural language processing (NLP) faces challenges when it encounters a word that is not categorized. These types of words are called out-of-vocabulary words (OOV). They are often the subject of representation, local slang, or typing mistakes. These types of content have grown exponentially as the Internet has popularized, making people interact more assiduously through texting. Given the importance of this subject, we present three OOV classification models based on deep learning using a corpus with words in Portuguese as a case study. These models are bidirectional simple recurrent neural networks (RNN), short-term long memory (LSTM), and gated recurrent units (GRU). The purpose is to enable the system to recognize the embedding of OOV and place them in a vector space. In addition, the meaning of the words was verified using cosine similarity. The results of LSTM are promising for identifying OOV and generating semantically similar words. The model can be used in pre-processing pipelines for user-generated content analysis, adding more value to social media studies.