Towards a LLM-based intelligent system for detecting propaganda within textual content

Abstract

Large Language Models (LLMs) have emerged as versatile and powerful tools for a wide array of natural language processing tasks, ranging from text generation to semantic comprehension. Among their diverse applications, LLMs exhibit significant potential in detecting propaganda. This work presents a computational approach for identifying propaganda techniques within textual content, leveraging both proprietary and open-source LLMs. The approach not only detects the presence of propaganda but also identifies specific parts of the text where these techniques are employed. Central to this methodology is the careful selection of LLMs and the application of advanced prompting strategies, including role-playing, reduced context windowing, few-shot learning, and chain-of-thought reasoning, to enhance prompt design and model performance. The effectiveness of the proposed approach was assessed through quantitative metrics. Additionally, an LLM-based intelligent system implementing the approach was developed and described in terms of its components and functionalities. This system, realized as a software prototype, was evaluated in SemEval 2020 Task 11 news articles, showcasing notable improvements over state-of-the-art methods in propaganda detection.