Content Moderation in Social Media: The Characteristics, Degree, and Efficiency of User Engagement

Abstract

Social media emerge as common platforms for knowledge sharing/exchange in online communities. Meanwhile, they also become a hotbed for the diffusion of misinformation. Content moderation is one of the measures for preventing the distribution of misinformation. Despite the increasing research attention to content moderation, the role of user engagement in content moderation remains significantly understudied. It is unclear how different characteristics and degrees of user engagement in social media might impact the performance of content moderation. In addition, the efficiency of content moderation has not been addressed by prior studies. This study aims to fill these research gaps by investigating the characteristics of user engagement behavior in social media and developing automated models to support content moderation that leverage a state-of-the-art pre-trained model for text analysis. The evaluation results with Reddit data suggest that the directivity and temporal characteristics of user engagement have significant effects on the effectiveness of content moderation. Additionally, leveraging the entire history of user engagement tends to be inefficient or even impractical, yet our findings provide evidence and a guide for improving the efficiency of content moderation using user engagement data without compromising model effectiveness. Our findings have research and practical implications for the moderation and deterrence of misinformation in social media.