Lexical Normalization of Roman Urdu

Abstract

Roman Urdu is an informal form of writing the Urdu language which is written in Latin script. It is the language most widely used on the internet, social media, and text messaging by native Urdu speakers. The problem that arises with Roman Urdu is an inconsistent way of writing by different people. No standard rules are defined for writing Roman Urdu which makes it very difficult to perform Natural Language Processing. To overcome this issue, the text needs to be normalized to perform effective analysis. Therefore, this work provides a Roman Urdu dictionary that works as the foundation for processing Roman Urdu. It also proposes a model for the lexical normalization of Roman Urdu text.