MRBalance: A framework for enhancing event causality identification in multi-agent debates via role assignment

Abstract

The rapid development of large language models (LLMs) has advanced natural language processing by improving contextual understanding and generalization abilities. However, despite these advances, determining event causality remains a challenging task. When LLMs are applied to this task, they frequently exhibit significant inconsistencies in recognizing causal representations, resulting in the phenomenon known as causal hallucinations. Specifically, LLMs perform well in predicting events with causal relationships but struggle with events without such relationships, frequently failing to achieve balanced performance across different causal scenarios. In this study, we propose MRBalance, a novel framework that uses role-based multi-agent debates to improve event causality identification. Our method transforms the task into a single-choice question-answering task, prompting LLM-based agents to engage in structured debates and justify their answers using their unique role-based perspectives. In addition, we introduce a mechanism for optimizing team members that selects the best agents to participate in the next debate when the debate rounds are lengthy. Extensive experiments on two benchmark datasets demonstrate significant performance improvements, highlighting the effectiveness of MRBalance in reducing causal hallucinations and increasing robustness.