CoolTeD: A Web-based Collaborative Labeling Tool for the Textual Dataset

Abstract

High-quality labeled textual data are vital for automatic mining and analysis of massive textual data produced by software systems. Several tools have been designed to facilitate manual labeling of textual data on different levels of granularity. However, these tools neither aim to provide statistics and analysis of labeled textual data, nor support collaboration among the coders to reduce the time cost in manual labeling and enhance the quality of labeling results. In this paper, we developed a Web-based labeling tool named CoolTeD (available at http://williamsriver.cn) for collaborative labeling of the textual datasets. Specifically, CoolTeD can be used: (1) to label textual data from the perspective of requirements types based on ISO 25010, (2) to review the labeling results with different confidence levels and contradictory labels, (3) to automatically calculate Cohen's Kappa coefficient of multiple coders, and (4) to visualize the labeling results. The tool demo is available at https://youtu.be/xVkrB_Cs1J8