Conference on Empirical Methods in Natural Language Processing最新文献_第3页

Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection 掩盖代写者：对人工智能生成的学生作文检测进行对抗性评估

Conference on Empirical Methods in Natural Language Processing Pub Date : 2024-02-01 DOI: 10.18653/v1/2023.emnlp-main.644

Xinlin Peng, Ying Zhou, Ben He, Le Sun, Yingfei Sun

{"title":"Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection","authors":"Xinlin Peng, Ying Zhou, Ben He, Le Sun, Yingfei Sun","doi":"10.18653/v1/2023.emnlp-main.644","DOIUrl":"https://doi.org/10.18653/v1/2023.emnlp-main.644","url":null,"abstract":"Large language models (LLMs) have exhibited remarkable capabilities in text generation tasks. However, the utilization of these models carries inherent risks, including but not limited to plagiarism, the dissemination of fake news, and issues in educational exercises. Although several detectors have been proposed to address these concerns, their effectiveness against adversarial perturbations, specifically in the context of student essay writing, remains largely unexplored. This paper aims to bridge this gap by constructing AIG-ASAP, an AI-generated student essay dataset, employing a range of text perturbation methods that are expected to generate high-quality essays while evading detection. Through empirical experiments, we assess the performance of current AIGC detectors on the AIG-ASAP dataset. The results reveal that the existing detectors can be easily circumvented using straightforward automatic adversarial attacks. Specifically, we explore word substitution and sentence substitution perturbation methods that effectively evade detection while maintaining the quality of the generated essays. This highlights the urgent need for more accurate and robust methods to detect AI-generated student essays in the education domain.","PeriodicalId":505350,"journal":{"name":"Conference on Empirical Methods in Natural Language Processing","volume":"36 43","pages":"10406-10419"},"PeriodicalIF":0.0,"publicationDate":"2024-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139684225","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph ReasoningLM：在预训练语言模型中启用结构子图推理，用于知识图谱上的问题解答

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-12-30 DOI: 10.18653/v1/2023.emnlp-main.228

Jinhao Jiang, Kun Zhou, Wayne Xin Zhao, Yaliang Li, Ji-Rong Wen

{"title":"ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph","authors":"Jinhao Jiang, Kun Zhou, Wayne Xin Zhao, Yaliang Li, Ji-Rong Wen","doi":"10.18653/v1/2023.emnlp-main.228","DOIUrl":"https://doi.org/10.18653/v1/2023.emnlp-main.228","url":null,"abstract":"Question Answering over Knowledge Graph (KGQA) aims to seek answer entities for the natural language question from a large-scale Knowledge Graph~(KG). To better perform reasoning on KG, recent work typically adopts a pre-trained language model~(PLM) to model the question, and a graph neural network~(GNN) based module to perform multi-hop reasoning on the KG. Despite the effectiveness, due to the divergence in model architecture, the PLM and GNN are not closely integrated, limiting the knowledge sharing and fine-grained feature interactions. To solve it, we aim to simplify the above two-module approach, and develop a more capable PLM that can directly support subgraph reasoning for KGQA, namely ReasoningLM. In our approach, we propose a subgraph-aware self-attention mechanism to imitate the GNN for performing structured reasoning, and also adopt an adaptation tuning strategy to adapt the model parameters with 20,000 subgraphs with synthesized questions. After adaptation, the PLM can be parameter-efficient fine-tuned on downstream tasks. Experiments show that ReasoningLM surpasses state-of-the-art models by a large margin, even with fewer updated parameters and less training data. Our codes and data are publicly available at~url{https://github.com/RUCAIBox/ReasoningLM}.","PeriodicalId":505350,"journal":{"name":"Conference on Empirical Methods in Natural Language Processing","volume":" 45","pages":"3721-3735"},"PeriodicalIF":0.0,"publicationDate":"2023-12-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139141665","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Exploring the Sensitivity of LLMs' Decision-Making Capabilities: Insights from Prompt Variations and Hyperparameters 探索 LLM 决策能力的敏感性：提示变化和超参数的启示

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-12-29 DOI: 10.18653/v1/2023.findings-emnlp.241

Manikanta Loya, Divya Sinha, Richard Futrell

引用次数: 0

AdapterDistillation: Non-Destructive Task Composition with Knowledge Distillation 适配器蒸馏：利用知识蒸馏实现非破坏性任务组合

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-12-26 DOI: 10.18653/v1/2023.emnlp-industry.20

Junjie Wang, Yicheng Chen, Wangshu Zhang, Sen Hu, Teng Xu, Jing Zheng

引用次数: 0

Black-Box Tuning of Vision-Language Models with Effective Gradient Approximation 用有效梯度逼近法对视觉语言模型进行黑盒调谐

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-12-26 DOI: 10.18653/v1/2023.findings-emnlp.356

Zixian Guo, Yuxiang Wei, Ming Liu, Zhilong Ji, Jinfeng Bai, Yiwen Guo, Wangmeng Zuo

{"title":"Black-Box Tuning of Vision-Language Models with Effective Gradient Approximation","authors":"Zixian Guo, Yuxiang Wei, Ming Liu, Zhilong Ji, Jinfeng Bai, Yiwen Guo, Wangmeng Zuo","doi":"10.18653/v1/2023.findings-emnlp.356","DOIUrl":"https://doi.org/10.18653/v1/2023.findings-emnlp.356","url":null,"abstract":"Parameter-efficient fine-tuning (PEFT) methods have provided an effective way for adapting large vision-language models to specific tasks or scenarios. Typically, they learn a very small scale of parameters for pre-trained models in a white-box formulation, which assumes model architectures to be known and parameters to be accessible. However, large models are often not open-source due to considerations of preventing abuse or commercial factors, hence posing a barrier to the deployment of white-box PEFT methods. To alleviate the dependence on model accessibility, we introduce collaborative black-box tuning (CBBT) for both textual prompt optimization and output feature adaptation for black-box models. Specifically, considering that the backpropagation gradients are blocked, we approximate the gradients of textual prompts by analyzing the predictions with perturbed prompts. Secondly, a lightweight adapter is deployed over the output feature of the inaccessible model, further facilitating the model adaptation process. Empowered with these designs, our CBBT is extensively evaluated on eleven downstream benchmarks and achieves remarkable improvements compared to existing black-box VL adaptation methods. Code is released at https://github.com/guozix/cbbt.","PeriodicalId":505350,"journal":{"name":"Conference on Empirical Methods in Natural Language Processing","volume":"10 19","pages":"5356-5368"},"PeriodicalIF":0.0,"publicationDate":"2023-12-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139155680","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Argue with Me Tersely: Towards Sentence-Level Counter-Argument Generation 与我喋喋不休地争论：实现句子级反驳生成

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-12-21 DOI: 10.18653/v1/2023.emnlp-main.1039

Jiayu Lin, Rong Ye, Meng Han, Qi Zhang, Ruofei Lai, Xinyu Zhang, Zhao Cao, Xuanjing Huang, Zhongyu Wei

{"title":"Argue with Me Tersely: Towards Sentence-Level Counter-Argument Generation","authors":"Jiayu Lin, Rong Ye, Meng Han, Qi Zhang, Ruofei Lai, Xinyu Zhang, Zhao Cao, Xuanjing Huang, Zhongyu Wei","doi":"10.18653/v1/2023.emnlp-main.1039","DOIUrl":"https://doi.org/10.18653/v1/2023.emnlp-main.1039","url":null,"abstract":"Counter-argument generation -- a captivating area in computational linguistics -- seeks to craft statements that offer opposing views. While most research has ventured into paragraph-level generation, sentence-level counter-argument generation beckons with its unique constraints and brevity-focused challenges. Furthermore, the diverse nature of counter-arguments poses challenges for evaluating model performance solely based on n-gram-based metrics. In this paper, we present the ArgTersely benchmark for sentence-level counter-argument generation, drawing from a manually annotated dataset from the ChangeMyView debate forum. We also propose Arg-LlaMA for generating high-quality counter-argument. For better evaluation, we trained a BERT-based evaluator Arg-Judge with human preference data. We conducted comparative experiments involving various baselines such as LlaMA, Alpaca, GPT-3, and others. The results show the competitiveness of our proposed framework and evaluator in counter-argument generation tasks. Code and data are available at https://github.com/amazingljy1206/ArgTersely.","PeriodicalId":505350,"journal":{"name":"Conference on Empirical Methods in Natural Language Processing","volume":"39 2","pages":"16705-16720"},"PeriodicalIF":0.0,"publicationDate":"2023-12-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139167005","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Faithful Model Evaluation for Model-Based Metrics 基于模型度量的忠实模型评估

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-12-19 DOI: 10.18653/v1/2023.emnlp-main.464

Qian Hu, Palash Goyal, Rahul Gupta

引用次数: 0

Relation-Aware Question Answering for Heterogeneous Knowledge Graphs 异构知识图谱的关系感知问题解答

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-12-19 DOI: 10.18653/v1/2023.findings-emnlp.906

Haowei Du, Quzhe Huang, Chen Li, Chen Zhang, Yang Li, Dongyan Zhao

{"title":"Relation-Aware Question Answering for Heterogeneous Knowledge Graphs","authors":"Haowei Du, Quzhe Huang, Chen Li, Chen Zhang, Yang Li, Dongyan Zhao","doi":"10.18653/v1/2023.findings-emnlp.906","DOIUrl":"https://doi.org/10.18653/v1/2023.findings-emnlp.906","url":null,"abstract":"Multi-hop Knowledge Base Question Answering(KBQA) aims to find the answer entity in a knowledge graph (KG), which requires multiple steps of reasoning. Existing retrieval-based approaches solve this task by concentrating on the specific relation at different hops and predicting the intermediate entity within the reasoning path. During the reasoning process of these methods, the representation of relations are fixed but the initial relation representation may not be optimal. We claim they fail to utilize information from head-tail entities and the semantic connection between relations to enhance the current relation representation, which undermines the ability to capture information of relations in KGs. To address this issue, we construct a textbf{dual relation graph} where each node denotes a relation in the original KG (textbf{primal entity graph}) and edges are constructed between relations sharing same head or tail entities. Then we iteratively do primal entity graph reasoning, dual relation graph information propagation, and interaction between these two graphs. In this way, the interaction between entity and relation is enhanced, and we derive better entity and relation representations. Experiments on two public datasets, WebQSP and CWQ, show that our approach achieves a significant performance gain over the prior state-of-the-art. Our code is available on url{https://github.com/yanmenxue/RAH-KBQA}.","PeriodicalId":505350,"journal":{"name":"Conference on Empirical Methods in Natural Language Processing","volume":"234 ","pages":"13582-13592"},"PeriodicalIF":0.0,"publicationDate":"2023-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139171771","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multi-Granularity Information Interaction Framework for Incomplete Utterance Rewriting 不完整语篇重写的多粒度信息交互框架

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-12-19 DOI: 10.18653/v1/2023.findings-emnlp.168

Haowei Du, Dinghao Zhang, Chen Li, Yang Li, Dongyan Zhao

引用次数: 0

Muted: Multilingual Targeted Offensive Speech Identification and Visualization 缄默：多语言攻击性语音识别与可视化

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-12-18 DOI: 10.18653/v1/2023.emnlp-demo.19

Christoph Tillmann, Aashka Trivedi, Sara Rosenthal, Santosh Borse, Rong Zhang, Avirup Sil, Bishwaranjan Bhattacharjee

引用次数: 0