Conference on Empirical Methods in Natural Language Processing最新文献_第4页

HEAR: Hearing Enhanced Audio Response for Video-grounded Dialogue HEAR：用于视频对话的听力增强音频响应

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-12-15 DOI: 10.18653/v1/2023.findings-emnlp.797

Sunjae Yoon, Dahyun Kim, Eunseop Yoon, Hee Suk Yoon, Junyeong Kim, Chang Dong Yoo

引用次数: 0

Discovering Highly Influential Shortcut Reasoning: An Automated Template-Free Approach 发现高影响捷径推理：无模板自动方法

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-12-15 DOI: 10.18653/v1/2023.findings-emnlp.424

Daichi Haraguchi, Kiyoaki Shirai, Naoya Inoue, Natthawut Kertkeidkachorn

引用次数: 0

Arabic Mini-ClimateGPT : A Climate Change and Sustainability Tailored Arabic LLM 阿拉伯语微型气候GPT：气候变化与可持续发展定制阿拉伯语法学硕士课程

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-12-14 DOI: 10.18653/v1/2023.findings-emnlp.941

Sahal Shaji Mullappilly, Abdelrahman M. Shaker, Omkar Thawakar, Hisham Cholakkal, R. Anwer, Salman H. Khan, Fahad Shahbaz Khan

{"title":"Arabic Mini-ClimateGPT : A Climate Change and Sustainability Tailored Arabic LLM","authors":"Sahal Shaji Mullappilly, Abdelrahman M. Shaker, Omkar Thawakar, Hisham Cholakkal, R. Anwer, Salman H. Khan, Fahad Shahbaz Khan","doi":"10.18653/v1/2023.findings-emnlp.941","DOIUrl":"https://doi.org/10.18653/v1/2023.findings-emnlp.941","url":null,"abstract":"Climate change is one of the most significant challenges we face together as a society. Creating awareness and educating policy makers the wide-ranging impact of climate change is an essential step towards a sustainable future. Recently, Large Language Models (LLMs) like ChatGPT and Bard have shown impressive conversational abilities and excel in a wide variety of NLP tasks. While these models are close-source, recently alternative open-source LLMs such as Stanford Alpaca and Vicuna have shown promising results. However, these open-source models are not specifically tailored for climate related domain specific information and also struggle to generate meaningful responses in other languages such as, Arabic. To this end, we propose a light-weight Arabic Mini-ClimateGPT that is built on an open-source LLM and is specifically fine-tuned on a conversational-style instruction tuning curated Arabic dataset Clima500-Instruct with over 500k instructions about climate change and sustainability. Further, our model also utilizes a vector embedding based retrieval mechanism during inference. We validate our proposed model through quantitative and qualitative evaluations on climate-related queries. Our model surpasses the baseline LLM in 88.3% of cases during ChatGPT-based evaluation. Furthermore, our human expert evaluation reveals an 81.6% preference for our model's responses over multiple popular open-source models. Our open-source demos, code-base and models are available here https://github.com/mbzuai-oryx/ClimateGPT.","PeriodicalId":505350,"journal":{"name":"Conference on Empirical Methods in Natural Language Processing","volume":"146 1","pages":"14126-14136"},"PeriodicalIF":0.0,"publicationDate":"2023-12-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139179824","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

ToViLaG: Your Visual-Language Generative Model is Also An Evildoer ToViLaG：你的视觉语言生成模型也是一个坏人

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-12-13 DOI: 10.18653/v1/2023.emnlp-main.213

Xinpeng Wang, Xiaoyuan Yi, Han Jiang, Shanlin Zhou, Zhihua Wei, Xing Xie

{"title":"ToViLaG: Your Visual-Language Generative Model is Also An Evildoer","authors":"Xinpeng Wang, Xiaoyuan Yi, Han Jiang, Shanlin Zhou, Zhihua Wei, Xing Xie","doi":"10.18653/v1/2023.emnlp-main.213","DOIUrl":"https://doi.org/10.18653/v1/2023.emnlp-main.213","url":null,"abstract":"Warning: this paper includes model outputs showing offensive content. Recent large-scale Visual-Language Generative Models (VLGMs) have achieved unprecedented improvement in multimodal image/text generation. However, these models might also generate toxic content, e.g., offensive text and pornography images, raising significant ethical risks. Despite exhaustive studies on toxic degeneration of language models, this problem remains largely unexplored within the context of visual-language generation. This work delves into the propensity for toxicity generation and susceptibility to toxic data across various VLGMs. For this purpose, we built ToViLaG, a dataset comprising 32K co-toxic/mono-toxic text-image pairs and 1K innocuous but evocative text that tends to stimulate toxicity. Furthermore, we propose WInToRe, a novel toxicity metric tailored to visual-language generation, which theoretically reflects different aspects of toxicity considering both input and output. On such a basis, we benchmarked the toxicity of a diverse spectrum of VLGMs and discovered that some models do more evil than expected while some are more vulnerable to infection, underscoring the necessity of VLGMs detoxification. Therefore, we develop an innovative bottleneck-based detoxification method. Our method could reduce toxicity while maintaining comparable generation quality, providing a promising initial solution to this line of research.","PeriodicalId":505350,"journal":{"name":"Conference on Empirical Methods in Natural Language Processing","volume":"63 5","pages":"3508-3533"},"PeriodicalIF":0.0,"publicationDate":"2023-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139181844","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Graph vs. Sequence: An Empirical Study on Knowledge Forms for Knowledge-Grounded Dialogue 图表与序列：以知识为基础的对话的知识形式实证研究

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-12-13 DOI: 10.18653/v1/2023.emnlp-main.982

Yizhe Yang, Heyan Huang, Yuhang Liu, Yang Gao

引用次数: 0

Large Language Models are Complex Table Parsers 大型语言模型是复杂的表解析器

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-12-13 DOI: 10.18653/v1/2023.emnlp-main.914

Bowen Zhao, Changkai Ji, Yuejie Zhang, Wen He, Yingwen Wang, Qing Wang, Rui Feng, Xiaobo Zhang

{"title":"Large Language Models are Complex Table Parsers","authors":"Bowen Zhao, Changkai Ji, Yuejie Zhang, Wen He, Yingwen Wang, Qing Wang, Rui Feng, Xiaobo Zhang","doi":"10.18653/v1/2023.emnlp-main.914","DOIUrl":"https://doi.org/10.18653/v1/2023.emnlp-main.914","url":null,"abstract":"With the Generative Pre-trained Transformer 3.5 (GPT-3.5) exhibiting remarkable reasoning and comprehension abilities in Natural Language Processing (NLP), most Question Answering (QA) research has primarily centered around general QA tasks based on GPT, neglecting the specific challenges posed by Complex Table QA. In this paper, we propose to incorporate GPT-3.5 to address such challenges, in which complex tables are reconstructed into tuples and specific prompt designs are employed for dialogues. Specifically, we encode each cell's hierarchical structure, position information, and content as a tuple. By enhancing the prompt template with an explanatory description of the meaning of each tuple and the logical reasoning process of the task, we effectively improve the hierarchical structure awareness capability of GPT-3.5 to better parse the complex tables. Extensive experiments and results on Complex Table QA datasets, i.e., the open-domain dataset HiTAB and the aviation domain dataset AIT-QA show that our approach significantly outperforms previous work on both datasets, leading to state-of-the-art (SOTA) performance.","PeriodicalId":505350,"journal":{"name":"Conference on Empirical Methods in Natural Language Processing","volume":"68 7-8","pages":"14786-14802"},"PeriodicalIF":0.0,"publicationDate":"2023-12-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139181366","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

COVID-19 Vaccine Misinformation in Middle Income Countries COVID-19 中等收入国家的疫苗误导

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-11-30 DOI: 10.48550/arXiv.2311.18195

Jongin Kim, Byeo Rhee Bak, Aditya Agrawal, Jiaxi Wu, Veronika J. Wirtz, Traci Hong, Derry Wijaya

{"title":"COVID-19 Vaccine Misinformation in Middle Income Countries","authors":"Jongin Kim, Byeo Rhee Bak, Aditya Agrawal, Jiaxi Wu, Veronika J. Wirtz, Traci Hong, Derry Wijaya","doi":"10.48550/arXiv.2311.18195","DOIUrl":"https://doi.org/10.48550/arXiv.2311.18195","url":null,"abstract":"This paper introduces a multilingual dataset of COVID-19 vaccine misinformation, consisting of annotated tweets from three middle-income countries: Brazil, Indonesia, and Nigeria. The expertly curated dataset includes annotations for 5,952 tweets, assessing their relevance to COVID-19 vaccines, presence of misinformation, and the themes of the misinformation. To address challenges posed by domain specificity, the low-resource setting, and data imbalance, we adopt two approaches for developing COVID-19 vaccine misinformation detection models: domain-specific pre-training and text augmentation using a large language model. Our best misinformation detection models demonstrate improvements ranging from 2.7 to 15.9 percentage points in macro F1-score compared to the baseline models. Additionally, we apply our misinformation detection models in a large-scale study of 19 million unlabeled tweets from the three countries between 2020 and 2022, showcasing the practical application of our dataset and models for detecting and analyzing vaccine misinformation in multiple countries and languages. Our analysis indicates that percentage changes in the number of new COVID-19 cases are positively associated with COVID-19 vaccine misinformation rates in a staggered manner for Brazil and Indonesia, and there are significant positive associations between the misinformation rates across the three countries.","PeriodicalId":505350,"journal":{"name":"Conference on Empirical Methods in Natural Language Processing","volume":"12 1","pages":"3903-3915"},"PeriodicalIF":0.0,"publicationDate":"2023-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139208944","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Evaluating the Rationale Understanding of Critical Reasoning in Logical Reading Comprehension 评估逻辑阅读理解中批判性推理的原理理解

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-11-30 DOI: 10.48550/arXiv.2311.18353

Akira Kawabata, Saku Sugawara

引用次数: 0

Introducing Rhetorical Parallelism Detection: A New Task with Datasets, Metrics, and Baselines 介绍修辞并行性检测：带数据集、度量标准和基线的新任务

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-11-30 DOI: 10.48550/arXiv.2312.00100

Stephen Lawrence Bothwell, Justin DeBenedetto, Theresa Crnkovich, Hildegund Müller, David Chiang

引用次数: 0

Unnatural Error Correction: GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text 非自然纠错：GPT-4 几乎能完美处理不自然的乱码文本

Conference on Empirical Methods in Natural Language Processing Pub Date : 2023-11-30 DOI: 10.48550/arXiv.2311.18805

Qi Cao, Takeshi Kojima, Yutaka Matsuo, Yusuke Iwasawa

{"title":"Unnatural Error Correction: GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text","authors":"Qi Cao, Takeshi Kojima, Yutaka Matsuo, Yusuke Iwasawa","doi":"10.48550/arXiv.2311.18805","DOIUrl":"https://doi.org/10.48550/arXiv.2311.18805","url":null,"abstract":"While Large Language Models (LLMs) have achieved remarkable performance in many tasks, much about their inner workings remains unclear. In this study, we present novel experimental insights into the resilience of LLMs, particularly GPT-4, when subjected to extensive character-level permutations. To investigate this, we first propose the Scrambled Bench, a suite designed to measure the capacity of LLMs to handle scrambled input, in terms of both recovering scrambled sentences and answering questions given scrambled context. The experimental results indicate that most powerful LLMs demonstrate the capability akin to typoglycemia, a phenomenon where humans can understand the meaning of words even when the letters within those words are scrambled, as long as the first and last letters remain in place. More surprisingly, we found that only GPT-4 nearly flawlessly processes inputs with unnatural errors, even under the extreme condition, a task that poses significant challenges for other LLMs and often even for humans. Specifically, GPT-4 can almost perfectly reconstruct the original sentences from scrambled ones, decreasing the edit distance by 95%, even when all letters within each word are entirely scrambled. It is counter-intuitive that LLMs can exhibit such resilience despite severe disruption to input tokenization caused by scrambled text.","PeriodicalId":505350,"journal":{"name":"Conference on Empirical Methods in Natural Language Processing","volume":"419 ","pages":"8898-8913"},"PeriodicalIF":0.0,"publicationDate":"2023-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139203382","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0