Statistical Tool for Arabic Text

Abstract

We present here a statistical tool dedicated to the Arabic language. This statistical tool uses encoding from the Unicode standard; the tool was programmed in the MATLAB environment. The statistical processing of the Arabic language constitutes a fundamental step for the realization and analysis of Arabic language corpora dedicated to various fields of application such as: the field of speech synthesis, speech recognition field, and the field of natural language processing. Our system which generates the statistical results related to the Arabic text is essentially based as input on a sequence of the diacritized Arabic text. The latter is transformed into data coded according to the Unicode standard so that the statistical rules base that we have developed can process it. The statistical tool developed provides useful information related to the treated Arabic text such as: number of words, occurrence frequency of each grapheme, and occurrence frequency of syllables "CV/CVV/CVC".