Together we can do it! A roadmap to effectively tackle propaganda-related tasks

Abstract

Purpose

In this paper, we address the need to study automatic propaganda detection to establish a course of action when faced with such a complex task. Although many isolated tasks have been proposed, a roadmap on how to best approach a new task from the perspective of text formality or the leverage of existing resources has not been explored yet.

Design/methodology/approach

We present a comprehensive study using several datasets on textual propaganda and different techniques to tackle it. We explore diverse collections with varied characteristics and analyze methodologies, from classic machine learning algorithms, to multi-task learning to utilize the available data in such models.

Findings

Our results show that transformer-based approaches are the best option with high-quality collections, and emotionally enriched inputs improve the results for Twitter content. Additionally, MTL achieves the best results in two of the five scenarios we analyzed. Notably, in one of the scenarios, the model achieves an F1 score of 0.78, significantly surpassing the transformer baseline model’s F1 score of 0.68.

Research limitations/implications

After finding a positive impact when leveraging propaganda’s emotional content, we propose further research into exploiting other complex dimensions, such as moral issues or logical reasoning.

Originality/value

Based on our findings, we provide a roadmap for tackling propaganda-related tasks, depending on the types of training data available and the task to solve. This includes the application of MTL, which has yet to be fully exploited in propaganda detection.