Annual Meeting of the Association for Computational Linguistics最新文献_第7页

RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbations 表QA对人类注释对抗性扰动鲁棒性的系统研究

Annual Meeting of the Association for Computational Linguistics Pub Date : 2023-06-25 DOI: 10.48550/arXiv.2306.14321

Yilun Zhao, Chen Zhao, Linyong Nan, Zhenting Qi, Wenlin Zhang, Xiangru Tang, Boyu Mi, Dragomir R. Radev

引用次数: 4

Unsupervised Mapping of Arguments of Deverbal Nouns to Their Corresponding Verbal Labels 述义名词论元到相应词性标签的无监督映射

Annual Meeting of the Association for Computational Linguistics Pub Date : 2023-06-24 DOI: 10.48550/arXiv.2306.13922

A. Weinstein, Yoav Goldberg

引用次数: 0

Class-Incremental Learning based on Label Generation 基于标签生成的类增量学习

Annual Meeting of the Association for Computational Linguistics Pub Date : 2023-06-22 DOI: 10.48550/arXiv.2306.12619

Yijia Shao, Yiduo Guo, Dongyan Zhao, Bin Liu

引用次数: 0

Feature Interactions Reveal Linguistic Structure in Language Models 特征交互揭示语言模型中的语言结构

Annual Meeting of the Association for Computational Linguistics Pub Date : 2023-06-21 DOI: 10.48550/arXiv.2306.12181

Jaap Jumelet, Willem H. Zuidema

{"title":"Feature Interactions Reveal Linguistic Structure in Language Models","authors":"Jaap Jumelet, Willem H. Zuidema","doi":"10.48550/arXiv.2306.12181","DOIUrl":"https://doi.org/10.48550/arXiv.2306.12181","url":null,"abstract":"We study feature interactions in the context of feature attribution methods for post-hoc interpretability. In interpretability research, getting to grips with feature interactions is increasingly recognised as an important challenge, because interacting features are key to the success of neural networks. Feature interactions allow a model to build up hierarchical representations for its input, and might provide an ideal starting point for the investigation into linguistic structure in language models. However, uncovering the exact role that these interactions play is also difficult, and a diverse range of interaction attribution methods has been proposed. In this paper, we focus on the question which of these methods most faithfully reflects the inner workings of the target models. We work out a grey box methodology, in which we train models to perfection on a formal language classification task, using PCFGs. We show that under specific configurations, some methods are indeed able to uncover the grammatical rules acquired by a model. Based on these findings we extend our evaluation to a case study on language models, providing novel insights into the linguistic structure that these models have acquired.","PeriodicalId":352845,"journal":{"name":"Annual Meeting of the Association for Computational Linguistics","volume":"130 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124250683","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Explicit Syntactic Guidance for Neural Text Generation 神经文本生成的显式句法指导

Annual Meeting of the Association for Computational Linguistics Pub Date : 2023-06-20 DOI: 10.48550/arXiv.2306.11485

Yafu Li, Leyang Cui, Jianhao Yan, Yongjng Yin, Wei Bi, Shuming Shi, Yue Zhang

引用次数: 1

The Ecological Fallacy in Annotation: Modeling Human Label Variation goes beyond Sociodemographics 注释中的生态谬误:人类标签变化的建模超越了社会人口统计学

Annual Meeting of the Association for Computational Linguistics Pub Date : 2023-06-20 DOI: 10.48550/arXiv.2306.11559

Matthias Orlikowski, Paul Röttger, P. Cimiano, Dirk Hovy Bielefeld University, U. Oxford, Computing Sciences Department, Bocconi University, Milan, Italy

{"title":"The Ecological Fallacy in Annotation: Modeling Human Label Variation goes beyond Sociodemographics","authors":"Matthias Orlikowski, Paul Röttger, P. Cimiano, Dirk Hovy Bielefeld University, U. Oxford, Computing Sciences Department, Bocconi University, Milan, Italy","doi":"10.48550/arXiv.2306.11559","DOIUrl":"https://doi.org/10.48550/arXiv.2306.11559","url":null,"abstract":"Many NLP tasks exhibit human label variation, where different annotators give different labels to the same texts. This variation is known to depend, at least in part, on the sociodemographics of annotators. Recent research aims to model individual annotator behaviour rather than predicting aggregated labels, and we would expect that sociodemographic information is useful for these models. On the other hand, the ecological fallacy states that aggregate group behaviour, such as the behaviour of the average female annotator, does not necessarily explain individual behaviour. To account for sociodemographics in models of individual annotator behaviour, we introduce group-specific layers to multi-annotator models. In a series of experiments for toxic content detection, we find that explicitly accounting for sociodemographic attributes in this way does not significantly improve model performance. This result shows that individual annotation behaviour depends on much more than just sociodemographics.","PeriodicalId":352845,"journal":{"name":"Annual Meeting of the Association for Computational Linguistics","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128493319","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Did the Models Understand Documents? Benchmarking Models for Language Understanding in Document-Level Relation Extraction 模型理解文档吗?文档级关系抽取中语言理解的基准模型

Annual Meeting of the Association for Computational Linguistics Pub Date : 2023-06-20 DOI: 10.48550/arXiv.2306.11386

Haotian Chen, Bingsheng Chen, Xiangdong Zhou

{"title":"Did the Models Understand Documents? Benchmarking Models for Language Understanding in Document-Level Relation Extraction","authors":"Haotian Chen, Bingsheng Chen, Xiangdong Zhou","doi":"10.48550/arXiv.2306.11386","DOIUrl":"https://doi.org/10.48550/arXiv.2306.11386","url":null,"abstract":"Document-level relation extraction (DocRE) attracts more research interest recently. While models achieve consistent performance gains in DocRE, their underlying decision rules are still understudied: Do they make the right predictions according to rationales? In this paper, we take the first step toward answering this question and then introduce a new perspective on comprehensively evaluating a model.Specifically, we first conduct annotations to provide the rationales considered by humans in DocRE. Then, we conduct investigations and discover the fact that: In contrast to humans, the representative state-of-the-art (SOTA) models in DocRE exhibit different reasoning processes. Through our proposed RE-specific attacks, we next demonstrate that the significant discrepancy in decision rules between models and humans severely damages the robustness of models. After that, we introduce mean average precision (MAP) to evaluate the understanding and reasoning capabilities of models. According to the extensive experimental results, we finally appeal to future work to consider evaluating the understanding ability of models because the improved ability renders models more trustworthy and robust to be deployed in real-world scenarios. We make our annotations and code publicly available.","PeriodicalId":352845,"journal":{"name":"Annual Meeting of the Association for Computational Linguistics","volume":"78 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116538816","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

CATS: A Pragmatic Chinese Answer-to-Sequence Dataset with Large Scale and High Quality CATS:一个大规模、高质量的汉语语用答案序列数据集

Annual Meeting of the Association for Computational Linguistics Pub Date : 2023-06-20 DOI: 10.48550/arXiv.2306.11477

Liang Li, Ruiying Geng, Chengyang Fang, Bing Li, Can Ma, Rongyu Cao, Binhua Li, Fei Huang, Yongbin Li

引用次数: 0

Jamp: Controlled Japanese Temporal Inference Dataset for Evaluating Generalization Capacity of Language Models 用于评估语言模型泛化能力的受控日语时间推理数据集

Annual Meeting of the Association for Computational Linguistics Pub Date : 2023-06-19 DOI: 10.48550/arXiv.2306.10727

Tomoki Sugimoto, Yasumasa Onoe, Hitomi Yanaka

引用次数: 0

Unsupervised Open-domain Keyphrase Generation 无监督开放域关键字生成

Annual Meeting of the Association for Computational Linguistics Pub Date : 2023-06-19 DOI: 10.48550/arXiv.2306.10755

Lam Thanh Do, Pritom Saha Akash, K. Chang

引用次数: 0