Exploiting Ensemble of Transformer Models for Detecting Informative Tweets

Abstract

Microblogging platforms especially Twitter is considered as one of the prominent medium of getting user-generated information. Millions of tweets were posted daily during COVID-19 pandemic days and the rate increases gradually. Tweets include a wide range of information including healthcare information, recent cases, and vaccination updates. This information helps individuals stay informed about the situation and assists safety personnel in making decisions. Apart from these, large amounts of propaganda and misinformation have spread on Twitter during this period. The impact of this infodemic is multifarious. Therefore, it is considered a formidable task to determine whether a tweet related to COVID-19 is informative or uninformative. However, the noisy and nonformal nature of tweets makes it difficult to determine the tweets’ informativeness. In this paper, we propose an approach that exploits the benefits of finetuned transformer models for informative tweet identification. Upon extracting features from pre-trained COVID-Twitter-BERT and RoBERTa models, we leverage the stacked embedding technique to combine them. The features are then fed to a BiLSTM module to learn the contextual dimension effectively. Finally, a simple feed-forward linear architecture is employed to obtain the predicted label. Experimental result on WNUT-2020 benchmark informative tweet detection dataset demonstrates the potency of our method over various state-of-the-art approaches.