Universal syllable tokeniser for language identification

Abstract

Phone recognition followed by language modeling gives good performance for language identification (LID). The requirement of labeled speech corpora makes it less appealing to build LID system. An alternative scalable approach is to build LID system that does not require annotated speech database. In this paper, we have compared two such LID systems namely Gaussian Mixture Model (GMM) tokeniser and syllable based LID system. The phonotactics of GMM and syllable based system are captured by GMM cluster indices and syllable tokens respectively. We propose the use of universal syllable models in building the LID systems and then deriving the uni-gram syllable statistics from this model. Experimental results on the OGI 1992 multilingual speech corpus show that syllable based LID system performs significantly better than the GMM Tokeniser system.