Describing rumours: a comparative evaluation of two handcrafted representations for rumour detection

Abstract

Nowadays, people use more and more social media as a source of information, leading to an increased and uncontrolled spread of misinformation. For this reason, tools to detect unverified and instrumentally relevant news, named as rumours, are necessary. In this work we compare two state-of-the-art handcrafted representations, namely User-Network and Social-Content, designed for developing machine learning-based rumour detection systems, in order to analyse which descriptors best capture the information hidden in unknown rumours. To this end we set up an experimental assessment implementing a Leave-One-Topic-Out evaluation on 8 different topics retrieved from Twitter social microblog. The results obtained for both representations are low as we designed a simple and non optimised pipeline for a fair comparison. Besides this, we were able to find out that the User-Network set of feature results more stable to topic changes. As a further contribution, we introduce two new datasets labelled for rumour detection task on Twitter.