Robust Morphologic Analyzer for Highly Inflected Languages

Abstract

We present a multilingual robust morphologic tagger and tokenizer for highly inflected languages like Spanish, with efficient spell correction and ‘sound-like’ word inference, obtaining some semantic extraction even on parasynthetic and unknown words. This algorithm combines rules, statistical best-affix-fit along with a language estimator. A rich flag set controls the internal behaviour. The system has been designed for efficiency and low memory footprint, using data structures based on simple available affixing rules. Our system, packed with a Spanish dictionary of 83k lemmas and 5k rules, recognizes 2.2M exact words, the guessing word-space is many times this much.