A robust and scalable framework for detecting self-reported illness from twitter

Early detection of onset and outbreak of infectious diseases has paramount importance in containing such diseases before they turn into epidemics. The incredible growth in popularity and spatial resolution of coverage have made micro-blogging sites like Twitter a promising source of information for assessing the evolution of intensity of such diseases within a locality. However, identifying tweets with self-reported illness from other `disease related' tweets is important for avoiding false alarms. In this research, our endeavor is to segregate the tweets all of which fall under the general category of `disease related'. By using relatively very small training set and modifying the conventional n-gram feature selection method, we could isolate tweets reporting individual's illness with around 88.7% precision.