Language detection using multinomial naïve bayes algorithm

Abstract

In this multilingual world, automatic detection of written or spoken language using Language Identification (LID) technology is a boon in the global communication with people using different languages in different countries. For simplicity and for the purpose of this research, the process of automatically identifying the language(s) from a document is thought of as LID. Lot of ongoing research projects are in the field of Natural Language Processing (NLP) that uses LID as a part of NLP. This field exploits several algorithms evolved in the field of computer science, individually or in combination to achieve accuracy in identifying a language. Among the different approaches adopted in LID,NaïveBayes Classification n-gram text processing seems to be promising.This paper proposes the concept for categorising multiple language texts using Naïve Bayesian algorithms using Machine Learning approaches. Using techniques from existing researches, this paper proposes a way to recognize multilingual documents and calculate the relative proportions of these languages.