Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Tutorials最新文献

筛选

英文中文

A Tutorial on Evaluation Metrics used in Natural Language Generation 自然语言生成中使用的评价指标教程

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Tutorials Pub Date : 2021-06-01 DOI: 10.18653/V1/2021.NAACL-TUTORIALS.4

Mitesh M. Khapra, Ananya B. Sai

{"title":"A Tutorial on Evaluation Metrics used in Natural Language Generation","authors":"Mitesh M. Khapra, Ananya B. Sai","doi":"10.18653/V1/2021.NAACL-TUTORIALS.4","DOIUrl":"https://doi.org/10.18653/V1/2021.NAACL-TUTORIALS.4","url":null,"abstract":"The advent of Deep Learning and the availability of large scale datasets has accelerated research on Natural Language Generation with a focus on newer tasks and better models. With such rapid progress, it is vital to assess the extent of scientific progress made and identify the areas/components that need improvement. To accomplish this in an automatic and reliable manner, the NLP community has actively pursued the development of automatic evaluation metrics. Especially in the last few years, there has been an increasing focus on evaluation metrics, with several criticisms of existing metrics and proposals for several new metrics. This tutorial presents the evolution of automatic evaluation metrics to their current state along with the emerging trends in this field by specifically addressing the following questions: (i) What makes NLG evaluation challenging? (ii) Why do we need automatic evaluation metrics? (iii) What are the existing automatic evaluation metrics and how can they be organised in a coherent taxonomy? (iv) What are the criticisms and shortcomings of existing metrics? (v) What are the possible future directions of research?","PeriodicalId":420993,"journal":{"name":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Tutorials","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126947720","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

Beyond Paragraphs: NLP for Long Sequences 超越段落:长序列的NLP

Iz Beltagy, Arman Cohan, Hannaneh Hajishirzi, Sewon Min, Matthew E. Peters

引用次数: 5

Crowdsourcing Natural Language Data at Scale: A Hands-On Tutorial 大规模众包自然语言数据:实践教程

Alexey Drutsa, Dmitry Ustalov, Valentina Fedorova, Olga Megorskaya, Daria Baidakova

引用次数: 6