发布求助
文献互助 智能选刊 最新文献

Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)最新文献

筛选
英文 中文
System Description for the CommonGen task with the POINTER model 使用POINTER模型的commonen任务的系统描述
Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021) Pub Date : 1900-01-01 DOI: 10.18653/v1/2021.gem-1.15
Anna Shvets
{"title":"System Description for the CommonGen task with the POINTER model","authors":"Anna Shvets","doi":"10.18653/v1/2021.gem-1.15","DOIUrl":"https://doi.org/10.18653/v1/2021.gem-1.15","url":null,"abstract":"In a current experiment we were testing CommonGen dataset for structure-to-text task from GEM living benchmark with the constraint based POINTER model. POINTER represents a hybrid architecture, combining insertion-based and transformer paradigms, predicting the token and the insertion position at the same time. The text is therefore generated gradually in a parallel non-autoregressive manner, given the set of keywords. The pretrained model was fine-tuned on a training split of the CommonGen dataset and the generation result was compared to the validation and challenge splits. The received metrics outputs, which measure lexical equivalence, semantic similarity and diversity, are discussed in details in a present system description.","PeriodicalId":431658,"journal":{"name":"Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128405737","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
NUIG-DSI’s submission to The GEM Benchmark 2021 NUIG-DSI提交给创业板基准2021
Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021) Pub Date : 1900-01-01 DOI: 10.18653/v1/2021.gem-1.13
Nivranshu Pasricha, Mihael Arcan, P. Buitelaar
{"title":"NUIG-DSI’s submission to The GEM Benchmark 2021","authors":"Nivranshu Pasricha, Mihael Arcan, P. Buitelaar","doi":"10.18653/v1/2021.gem-1.13","DOIUrl":"https://doi.org/10.18653/v1/2021.gem-1.13","url":null,"abstract":"This paper describes the submission by NUIG-DSI to the GEM benchmark 2021. We participate in the modeling shared task where we submit outputs on four datasets for data-to-text generation, namely, DART, WebNLG (en), E2E and CommonGen. We follow an approach similar to the one described in the GEM benchmark paper where we use the pre-trained T5-base model for our submission. We train this model on additional monolingual data where we experiment with different masking strategies specifically focused on masking entities, predicates and concepts as well as a random masking strategy for pre-training. In our results we find that random masking performs the best in terms of automatic evaluation metrics, though the results are not statistically significantly different compared to other masking strategies.","PeriodicalId":431658,"journal":{"name":"Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)","volume":"115 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117276439","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
SimpleNER Sentence Simplification System for GEM 2021 SimpleNER GEM 2021句子简化系统
Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021) Pub Date : 1900-01-01 DOI: 10.18653/v1/2021.gem-1.14
KV Aditya Srivatsa, Monil Gokani, Manish Shrivastava
{"title":"SimpleNER Sentence Simplification System for GEM 2021","authors":"KV Aditya Srivatsa, Monil Gokani, Manish Shrivastava","doi":"10.18653/v1/2021.gem-1.14","DOIUrl":"https://doi.org/10.18653/v1/2021.gem-1.14","url":null,"abstract":"This paper describes SimpleNER, a model developed for the sentence simplification task at GEM-2021. Our system is a monolingual Seq2Seq Transformer architecture that uses control tokens pre-pended to the data, allowing the model to shape the generated simplifications according to user desired attributes. Additionally, we show that NER-tagging the training data before use helps stabilize the effect of the control tokens and significantly improves the overall performance of the system. We also employ pretrained embeddings to reduce data sparsity and allow the model to produce more generalizable outputs.","PeriodicalId":431658,"journal":{"name":"Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127699895","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Human Perception in Natural Language Generation 自然语言生成中的人类感知
Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021) Pub Date : 1900-01-01 DOI: 10.18653/v1/2021.gem-1.2
Lorenzo De Mattei, Huiyuan Lai, F. Dell’Orletta, M. Nissim
{"title":"Human Perception in Natural Language Generation","authors":"Lorenzo De Mattei, Huiyuan Lai, F. Dell’Orletta, M. Nissim","doi":"10.18653/v1/2021.gem-1.2","DOIUrl":"https://doi.org/10.18653/v1/2021.gem-1.2","url":null,"abstract":"We ask subjects whether they perceive as human-produced a bunch of texts, some of which are actually human-written, while others are automatically generated. We use this data to fine-tune a GPT-2 model to push it to generate more human-like texts, and observe that this fine-tuned model produces texts that are indeed perceived more human-like than the original model. Contextually, we show that our automatic evaluation strategy well correlates with human judgements. We also run a linguistic analysis to unveil the characteristics of human- vs machine-perceived language.","PeriodicalId":431658,"journal":{"name":"Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124675960","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
Semantic Similarity Based Evaluation for Abstractive News Summarization 基于语义相似度的新闻文摘评价
Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021) Pub Date : 1900-01-01 DOI: 10.18653/v1/2021.gem-1.3
Figen Beken Fikri, Kemal Oflazer, B. Yanikoglu
{"title":"Semantic Similarity Based Evaluation for Abstractive News Summarization","authors":"Figen Beken Fikri, Kemal Oflazer, B. Yanikoglu","doi":"10.18653/v1/2021.gem-1.3","DOIUrl":"https://doi.org/10.18653/v1/2021.gem-1.3","url":null,"abstract":"ROUGE is a widely used evaluation metric in text summarization. However, it is not suitable for the evaluation of abstractive summarization systems as it relies on lexical overlap between the gold standard and the generated summaries. This limitation becomes more apparent for agglutinative languages with very large vocabularies and high type/token ratios. In this paper, we present semantic similarity models for Turkish and apply them as evaluation metrics for an abstractive summarization task. To achieve this, we translated the English STSb dataset into Turkish and presented the first semantic textual similarity dataset for Turkish as well. We showed that our best similarity models have better alignment with average human judgments compared to ROUGE in both Pearson and Spearman correlations.","PeriodicalId":431658,"journal":{"name":"Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)","volume":"94 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128843073","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 10
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整，为了账户安全，请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们：info@booksci.cn Book学术提供免费学术资源搜索服务，方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号：481959085
Book学术官方微信