Identifying arguments within a text: Categorizing errors and their impact in arguments' relation prediction

Abstract

The automatic identification of argument units within a text is a crucial task, as it is the first step that should be performed by an end-to-end argument mining system. In this work, we propose an approach for categorizing errors in predicted argument units, which allows the evaluation of segmentation models from an argumentative perspective. We assess the ability of several models to generalize knowledge across different text domains and, through the proposed categorization, we show differences in their behavior that may not be noticeable using standard classification metrics. Furthermore, we assess how the errors in predicted argument units impact on a task that rely on accurate unit identification, an aspect that has not been studied in previous research, and that helps to evaluate the usability of an imperfect segmentation model beyond the segmentation task itself.