Transactions of the Association for Computational Linguistics最新文献_第7页

Transparency Helps Reveal When Language Models Learn Meaning 透明度有助于揭示语言模型何时学习意义

IF 10.9 1区计算机科学

Transactions of the Association for Computational Linguistics Pub Date : 2022-10-14 DOI: 10.1162/tacl_a_00565

Zhaofeng Wu, Will Merrill, Hao Peng, Iz Beltagy, Noah A. Smith

引用次数: 2

Explainable Abuse Detection as Intent Classification and Slot Filling 可解释的滥用检测:意图分类和槽填充

IF 10.9 1区计算机科学

Transactions of the Association for Computational Linguistics Pub Date : 2022-10-06 DOI: 10.1162/tacl_a_00527

Agostina Calabrese, Björn Ross, Mirella Lapata

引用次数: 4

Domain-Specific Word Embeddings with Structure Prediction 具有结构预测的领域特定词嵌入

IF 10.9 1区计算机科学

Transactions of the Association for Computational Linguistics Pub Date : 2022-10-06 DOI: 10.1162/tacl_a_00538

Stephanie Brandl, D. Lassner, A. Baillot, S. Nakajima

{"title":"Domain-Specific Word Embeddings with Structure Prediction","authors":"Stephanie Brandl, D. Lassner, A. Baillot, S. Nakajima","doi":"10.1162/tacl_a_00538","DOIUrl":"https://doi.org/10.1162/tacl_a_00538","url":null,"abstract":"Complementary to finding good general word embeddings, an important question for representation learning is to find dynamic word embeddings, for example, across time or domain. Current methods do not offer a way to use or predict information on structure between sub-corpora, time or domain and dynamic embeddings can only be compared after post-alignment. We propose novel word embedding methods that provide general word representations for the whole corpus, domain- specific representations for each sub-corpus, sub-corpus structure, and embedding alignment simultaneously. We present an empirical evaluation on New York Times articles and two English Wikipedia datasets with articles on science and philosophy. Our method, called Word2Vec with Structure Prediction (W2VPred), provides better performance than baselines in terms of the general analogy tests, domain-specific analogy tests, and multiple specific word embedding evaluations as well as structure prediction performance when no structure is given a priori. As a use case in the field of Digital Humanities we demonstrate how to raise novel research questions for high literature from the German Text Archive.","PeriodicalId":33559,"journal":{"name":"Transactions of the Association for Computational Linguistics","volume":"11 1","pages":"320-335"},"PeriodicalIF":10.9,"publicationDate":"2022-10-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43780471","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering 面向开放领域问答的检索增强生成(RAG)模型的领域适应性改进

IF 10.9 1区计算机科学

Transactions of the Association for Computational Linguistics Pub Date : 2022-10-06 DOI: 10.1162/tacl_a_00530

Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, R. Rana, Suranga Nanayakkara

{"title":"Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering","authors":"Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, R. Rana, Suranga Nanayakkara","doi":"10.1162/tacl_a_00530","DOIUrl":"https://doi.org/10.1162/tacl_a_00530","url":null,"abstract":"Retrieval Augment Generation (RAG) is a recent advancement in Open-Domain Question Answering (ODQA). RAG has only been trained and explored with a Wikipedia-based external knowledge base and is not optimized for use in other specialized domains such as healthcare and news. In this paper, we evaluate the impact of joint training of the retriever and generator components of RAG for the task of domain adaptation in ODQA. We propose RAG-end2end, an extension to RAG that can adapt to a domain-specific knowledge base by updating all components of the external knowledge base during training. In addition, we introduce an auxiliary training signal to inject more domain-specific knowledge. This auxiliary signal forces RAG-end2end to reconstruct a given sentence by accessing the relevant information from the external knowledge base. Our novel contribution is that, unlike RAG, RAG-end2end does joint training of the retriever and generator for the end QA task and domain adaptation. We evaluate our approach with datasets from three domains: COVID-19, News, and Conversations, and achieve significant performance improvements compared to the original RAG model. Our work has been open-sourced through the HuggingFace Transformers library, attesting to our work’s credibility and technical consistency.","PeriodicalId":33559,"journal":{"name":"Transactions of the Association for Computational Linguistics","volume":"55 1","pages":"1-17"},"PeriodicalIF":10.9,"publicationDate":"2022-10-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"64440765","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 10

FRMT: A Benchmark for Few-Shot Region-Aware Machine Translation 少镜头区域感知机器翻译的基准

IF 10.9 1区计算机科学

Transactions of the Association for Computational Linguistics Pub Date : 2022-10-01 DOI: 10.1162/tacl_a_00568

Parker Riley, Timothy Dozat, Jan A. Botha, Xavier García, Dan Garrette, Jason Riesa, Orhan Firat, Noah Constant

引用次数: 6

Meta-Learning a Cross-lingual Manifold for Semantic Parsing 元学习——用于语义分析的跨语言流形

IF 10.9 1区计算机科学

Transactions of the Association for Computational Linguistics Pub Date : 2022-09-26 DOI: 10.1162/tacl_a_00533

Tom Sherborne, Mirella Lapata

引用次数: 7

OPAL: Ontology-Aware Pretrained Language Model for End-to-End Task-Oriented Dialogue 面向端到端任务对话的本体感知预训练语言模型

IF 10.9 1区计算机科学

Transactions of the Association for Computational Linguistics Pub Date : 2022-09-10 DOI: 10.1162/tacl_a_00534

Zhi Chen, Yuncong Liu, Lu Chen, Su Zhu, Mengyue Wu, Kai Yu

{"title":"OPAL: Ontology-Aware Pretrained Language Model for End-to-End Task-Oriented Dialogue","authors":"Zhi Chen, Yuncong Liu, Lu Chen, Su Zhu, Mengyue Wu, Kai Yu","doi":"10.1162/tacl_a_00534","DOIUrl":"https://doi.org/10.1162/tacl_a_00534","url":null,"abstract":"This paper presents an ontology-aware pretrained language model (OPAL) for end-to-end task-oriented dialogue (TOD). Unlike chit-chat dialogue models, task-oriented dialogue models fulfill at least two task-specific modules: Dialogue state tracker (DST) and response generator (RG). The dialogue state consists of the domain-slot-value triples, which are regarded as the user’s constraints to search the domain-related databases. The large-scale task-oriented dialogue data with the annotated structured dialogue state usually are inaccessible. It prevents the development of the pretrained language model for the task-oriented dialogue. We propose a simple yet effective pretraining method to alleviate this problem, which consists of two pretraining phases. The first phase is to pretrain on large-scale contextual text data, where the structured information of the text is extracted by the information extracting tool. To bridge the gap between the pretraining method and downstream tasks, we design two pretraining tasks: ontology-like triple recovery and next-text generation, which simulates the DST and RG, respectively. The second phase is to fine-tune the pretrained model on the TOD data. The experimental results show that our proposed method achieves an exciting boost and obtains competitive performance even without any TOD data on CamRest676 and MultiWOZ benchmarks.","PeriodicalId":33559,"journal":{"name":"Transactions of the Association for Computational Linguistics","volume":"11 1","pages":"68-84"},"PeriodicalIF":10.9,"publicationDate":"2022-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44597473","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

Investigating Reasons for Disagreement in Natural Language Inference 探究自然语言推理中分歧的原因

IF 10.9 1区计算机科学

Transactions of the Association for Computational Linguistics Pub Date : 2022-09-07 DOI: 10.1162/tacl_a_00523

Nan Jiang, M. Marneffe

引用次数: 19

Adapting to the Long Tail: A Meta-Analysis of Transfer Learning Research for Language Understanding Tasks. 适应长尾:语言理解任务迁移学习研究的元分析。

IF 10.9 1区计算机科学

Transactions of the Association for Computational Linguistics Pub Date : 2022-09-07 eCollection Date: 2022-10-01 DOI: 10.1162/tacl_a_00500

Aakanksha Naik, Jill Lehman, Carolyn Rosé

{"title":"Adapting to the Long Tail: A Meta-Analysis of Transfer Learning Research for Language Understanding Tasks.","authors":"Aakanksha Naik, Jill Lehman, Carolyn Rosé","doi":"10.1162/tacl_a_00500","DOIUrl":"https://doi.org/10.1162/tacl_a_00500","url":null,"abstract":"Natural language understanding (NLU) has made massive progress driven by large benchmarks, but benchmarks often leave a long tail of infrequent phenomena underrepresented. We reflect on the question: Have transfer learning methods sufficiently addressed the poor performance of benchmark-trained models on the long tail? We conceptualize the long tail using macro-level dimensions (underrepresented genres, topics, etc.), and perform a qualitative meta-analysis of 100 representative papers on transfer learning research for NLU. Our analysis asks three questions: (i) Which long tail dimensions do transfer learning studies target? (ii) Which properties of adaptation methods help improve performance on the long tail? (iii) Which methodological gaps have greatest negative impact on long tail performance? Our answers highlight major avenues for future research in transfer learning for the long tail. Lastly, using our meta-analysis framework, we perform a case study comparing the performance of various adaptation methods on clinical narratives, which provides interesting insights that may enable us to make progress along these future avenues.","PeriodicalId":33559,"journal":{"name":"Transactions of the Association for Computational Linguistics","volume":"10 ","pages":"956-980"},"PeriodicalIF":10.9,"publicationDate":"2022-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9590102/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"40667339","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Efficient Methods for Natural Language Processing: A Survey 自然语言处理的有效方法综述

IF 10.9 1区计算机科学

Transactions of the Association for Computational Linguistics Pub Date : 2022-08-31 DOI: 10.1162/tacl_a_00577

Marcos Vinícius Treviso, Tianchu Ji, Ji-Ung Lee, Betty van Aken, Qingqing Cao, Manuel R. Ciosici, Michael Hassid, Kenneth Heafield, Sara Hooker, Pedro Henrique Martins, André F. T. Martins, Peter Milder, Colin Raffel, Edwin Simpson, N. Slonim, Niranjan Balasubramanian, Leon Derczynski, Roy Schwartz

引用次数: 38