"Comments Matter and The More The Better!": Improving Rumor Detection with User Comments

Abstract

While many online platforms bring great benefits to their users by allowing user-generated content, they have also facilitated generation and spreading of harmful content such as rumors. Researcher have proposed different rumor detection methods based on features extracted from the original post and/or associated comments, but how comments affect the performance of such methods remains largely less understood. In this paper, we first propose a new BERT-based rumor detection method that can outperform other state-of-the-art methods, and then used it to study the role of comments in rumor detection. Our proposed method concatenates the original post and associated comments to form a single long text, which is then segmented into shorter chunks more suitable for BERT-based vectorization. Features extracted from all trunks are fed into a classifier based on an LSTM network or a transformer layer for the classification task. The experimental results on the PHEME and Ma-Weibo datasets proved the superior performance of our method. We conducted additional experiments on different settings of our proposed method to study different aspects of the role comments play in the rumor detection task. These additional experiments led to some very interesting findings, including the surprising result that fixed-length segmentation is better than natural segmentation, and the observation that including more comments can help improve the rumor detector’s performance. Some of these findings have profound operational implications for online platforms, e.g., commentators can contribute to rumor detection positively so online platforms can leverage the crowd intelligence to detect online rumors more effectively without applying overstrict content consensus policies.