访谈对话语料中话语意图标签的标注与评价

Proceedings of the 2021 5th International Conference on Natural Language Processing and Information Retrieval Pub Date : 2021-12-17 DOI:10.1145/3508230.3508236

M. Sasayama, Kazuyuki Matsumoto

{"title":"访谈对话语料中话语意图标签的标注与评价","authors":"M. Sasayama, Kazuyuki Matsumoto","doi":"10.1145/3508230.3508236","DOIUrl":null,"url":null,"abstract":"In this paper, we proposed the utterance intention tags for an interview dialogue corpus. We constructed an interview dialogue corpus with the tags we designed. Three or five annotators annotated the tags to the interview dialogue corpus (the total number of 49999 utterances) consisting of 30 dialogues. We conducted an evaluation experiment using Fleiss's kappa value to evaluate the reliability of the proposed tags. When three annotators annotated 18 different tags to the corpus, we obtained the kappa value of 0.55.","PeriodicalId":252146,"journal":{"name":"Proceedings of the 2021 5th International Conference on Natural Language Processing and Information Retrieval","volume":"300 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2021-12-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"Annotation and Evaluation of Utterance Intention Tag for Interview Dialogue Corpus\",\"authors\":\"M. Sasayama, Kazuyuki Matsumoto\",\"doi\":\"10.1145/3508230.3508236\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In this paper, we proposed the utterance intention tags for an interview dialogue corpus. We constructed an interview dialogue corpus with the tags we designed. Three or five annotators annotated the tags to the interview dialogue corpus (the total number of 49999 utterances) consisting of 30 dialogues. We conducted an evaluation experiment using Fleiss's kappa value to evaluate the reliability of the proposed tags. When three annotators annotated 18 different tags to the corpus, we obtained the kappa value of 0.55.\",\"PeriodicalId\":252146,\"journal\":{\"name\":\"Proceedings of the 2021 5th International Conference on Natural Language Processing and Information Retrieval\",\"volume\":\"300 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2021-12-17\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Proceedings of the 2021 5th International Conference on Natural Language Processing and Information Retrieval\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1145/3508230.3508236\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the 2021 5th International Conference on Natural Language Processing and Information Retrieval","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3508230.3508236","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}

引用次数: 1

摘要

在本文中，我们提出了一个访谈对话语料库的话语意图标签。我们用我们设计的标签构建了一个访谈对话语料库。三个或五个注释者将标签注释到由30个对话组成的访谈对话语料库(总共49999个话语)。我们使用Fleiss的kappa值进行了评估实验，以评估所提出标签的可靠性。当三个注释者对语料标注18个不同的标签时，我们得到kappa值为0.55。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Annotation and Evaluation of Utterance Intention Tag for Interview Dialogue Corpus

In this paper, we proposed the utterance intention tags for an interview dialogue corpus. We constructed an interview dialogue corpus with the tags we designed. Three or five annotators annotated the tags to the interview dialogue corpus (the total number of 49999 utterances) consisting of 30 dialogues. We conducted an evaluation experiment using Fleiss's kappa value to evaluate the reliability of the proposed tags. When three annotators annotated 18 different tags to the corpus, we obtained the kappa value of 0.55.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Proceedings of the 2021 5th International Conference on Natural Language Processing and Information Retrieval

自引率

0.00%

发文量