Automatic topic identification and classification of text messages in the SMSAll system

Abstract

This paper presents a way to identify topics and classify text messages in the SMSAll system, which is the Twitter of Pakistan (except over SMS). Among many challenges, one is to develop an unsupervised algorithm for text messages containing Urdu-English words written in roman letters. Still in 1-gram we are able to have 72%, 53% and 58% true positives for popular, medium and rare topics respectively and 48% and 40% true positives in 2 and 3-grams respectively.