Design of a POS tagger using conditional random fields for Malayalam

Abstract

Parts of Speech tagging, is a process of marking the words in a text as corresponding to a particular part of speech, based on its definition and context POS tagger plays an important role in Natural language applications like speech recognition, natural language parsing, information retrieval and extraction. This paper discusses architecture for designing a Part-Of-Speech (POS tagger for Malayalam language using Conditional Random Field (CRF). The experiments presented in this paper use an annotated corpus of 1028 sentences (11,315 words) and tagset consists of 100 tags. A trigram based tagging scheme is involved in the experiments. The proposed system is based on an empirical approach that models the human POS tagging processing more realistically than the existing systems, without compromising the efficiency and accuracy.