Findings of ACL. ACL最新文献_第2页

LADDER: Language-Driven Slice Discovery and Error Rectification in Vision Classifiers. 阶梯：视觉分类器中语言驱动的切片发现和错误纠正。

Findings of ACL. ACL Pub Date : 2025-07-01 DOI: 10.18653/v1/2025.findings-acl.1177

Shantanu Ghosh, Rayan Syed, Chenyu Wang, Vaibhav Choudhary, Binxu Li, Clare B Poynton, Shyam Visweswaran, Kayhan Batmanghelich

{"title":"LADDER: Language-Driven Slice Discovery and Error Rectification in Vision Classifiers.","authors":"Shantanu Ghosh, Rayan Syed, Chenyu Wang, Vaibhav Choudhary, Binxu Li, Clare B Poynton, Shyam Visweswaran, Kayhan Batmanghelich","doi":"10.18653/v1/2025.findings-acl.1177","DOIUrl":"10.18653/v1/2025.findings-acl.1177","url":null,"abstract":"Slice discovery refers to identifying systematic biases in the mistakes of pre-trained vision models. Current slice discovery methods in computer vision rely on converting input images into sets of attributes and then testing hypotheses about configurations of these precomputed attributes associated with elevated error patterns. However, such methods face several limitations: 1) they are restricted by the predefined attribute bank; 2) they lack the common sense reasoning and domain-specific knowledge often required for specialized fields e.g., radiology; 3) at best, they can only identify biases in image attributes while overlooking those introduced during preprocessing or data preparation. We hypothesize that bias-inducing variables leave traces in the form of language (e.g., logs), which can be captured as unstructured text. Thus, we introduce LADDER, which leverages the reasoning capabilities and latent domain knowledge of Large Language Models (LLMs) to generate hypotheses about these mistakes. Specifically, we project the internal activations of a pre-trained model into text using a retrieval approach and prompt the LLM to propose potential bias hypotheses. To detect biases from preprocessing pipelines, we convert the preprocessing data into text and prompt the LLM. Finally, LADDER generates pseudo-labels for each identified bias, thereby mitigating all biases without requiring expensive attribute annotations. Rigorous evaluations on 3 natural and 3 medical imaging datasets, 200+ classifiers, and 4 LLMs with varied architectures and pretraining strategies - demonstrate that LADDER consistently outperforms current methods.","PeriodicalId":521131,"journal":{"name":"Findings of ACL. ACL","volume":"2025 ","pages":"22935-22970"},"PeriodicalIF":0.0,"publicationDate":"2025-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12756946/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145902195","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Behavioral Analysis of Information Salience in Large Language Models. 大型语言模型中信息显著性的行为分析。

Findings of ACL. ACL Pub Date : 2025-07-01 DOI: 10.18653/v1/2025.findings-acl.1204

Jan Trienes, Jörg Schlötterer, Junyi Jessy Li, Christin Seifert

引用次数: 0

Too Big to Fail: Larger Language Models are Disproportionately Resilient to Induction of Dementia-Related Linguistic Anomalies. 太大而不能失败：更大的语言模型对痴呆相关语言异常的诱导具有不成比例的弹性。

Findings of ACL. ACL Pub Date : 2024-08-01 DOI: 10.18653/v1/2024.findings-acl.380

Changye Li, Zhecheng Sheng, Trevor Cohen, Serguei Pakhomov

{"title":"Too Big to Fail: Larger Language Models are Disproportionately Resilient to Induction of Dementia-Related Linguistic Anomalies.","authors":"Changye Li, Zhecheng Sheng, Trevor Cohen, Serguei Pakhomov","doi":"10.18653/v1/2024.findings-acl.380","DOIUrl":"10.18653/v1/2024.findings-acl.380","url":null,"abstract":"As artificial neural networks grow in complexity, understanding their inner workings becomes increasingly challenging, which is particularly important in healthcare applications. The intrinsic evaluation metrics of autoregressive neural language models (NLMs), perplexity (PPL), can reflect how \"surprised\" an NLM model is at novel input. PPL has been widely used to understand the behavior of NLMs. Previous findings show that changes in PPL when masking attention layers in pre-trained transformer-based NLMs reflect linguistic anomalies associated with Alzheimer's disease dementia. Building upon this, we explore a novel bidirectional attention head ablation method that exhibits properties attributed to the concepts of cognitive and brain reserve in human brain studies, which postulate that people with more neurons in the brain and more efficient processing are more resilient to neurodegeneration. Our results show that larger GPT-2 models require a disproportionately larger share of attention heads to be masked/ablated to display degradation of similar magnitude to masking in smaller models. These results suggest that the attention mechanism in transformer models may present an analogue to the notions of cognitive and brain reserve and could potentially be used to model certain aspects of the progression of neurodegenerative disorders and aging.","PeriodicalId":521131,"journal":{"name":"Findings of ACL. ACL","volume":"2024 ","pages":"6363-6377"},"PeriodicalIF":0.0,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12674208/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145679835","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

ELAD: Explanation-Guided Large Language Models Active Distillation. ELAD：解释引导的大语言模型主动蒸馏。

Findings of ACL. ACL Pub Date : 2024-08-01 DOI: 10.18653/v1/2024.findings-acl.264

Yifei Zhang, Bo Pan, Chen Ling, Yuntong Hu, Liang Zhao

{"title":"ELAD: Explanation-Guided Large Language Models Active Distillation.","authors":"Yifei Zhang, Bo Pan, Chen Ling, Yuntong Hu, Liang Zhao","doi":"10.18653/v1/2024.findings-acl.264","DOIUrl":"10.18653/v1/2024.findings-acl.264","url":null,"abstract":"The deployment and application of Large Language Models (LLMs) is hindered by their memory inefficiency, computational demands, and the high costs of API inferences. Traditional distillation methods, which transfer the capabilities of LLMs to smaller models, often fail to determine whether the knowledge has been sufficiently transferred, potentially resulting in high costs or incomplete distillation. In this paper, we propose an Explanation-Guided LLMs Active Distillation (ELAD) framework that employs an active learning strategy to optimize the balance between annotation costs and model performance. To improve the efficiency of sample selection, we introduce an explanation-guided sample selection method that identifies samples challenging its reasoning by exploiting uncertainties in reasoning explanation steps. Additionally, we present a customized LLM-annotated explanation revision technique where the teacher model detects and corrects flaws in the student model's reasoning. Our experiments across various reasoning datasets demonstrate that our framework significantly enhances the efficiency of LLMs knowledge distillation.","PeriodicalId":521131,"journal":{"name":"Findings of ACL. ACL","volume":"2024 ","pages":"4463-4475"},"PeriodicalIF":0.0,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13499443/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148805462","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

NoteChat: A Dataset of Synthetic Patient-Physician Conversations Conditioned on Clinical Notes. NoteChat：一个基于临床记录的合成医患对话数据集。

Findings of ACL. ACL Pub Date : 2024-08-01 DOI: 10.18653/v1/2024.findings-acl.901

Junda Wang, Zonghai Yao, Zhichao Yang, Huixue Zhou, Rumeng Li, Xun Wang, Yucheng Xu, Hong Yu

{"title":"NoteChat: A Dataset of Synthetic Patient-Physician Conversations Conditioned on Clinical Notes.","authors":"Junda Wang, Zonghai Yao, Zhichao Yang, Huixue Zhou, Rumeng Li, Xun Wang, Yucheng Xu, Hong Yu","doi":"10.18653/v1/2024.findings-acl.901","DOIUrl":"10.18653/v1/2024.findings-acl.901","url":null,"abstract":"We introduce NoteChat, a novel cooperative multi-agent framework leveraging Large Language Models (LLMs) to generate patient-physician dialogues. NoteChat embodies the principle that an ensemble of role-specific LLMs, through structured role-play and strategic prompting, can perform their assigned roles more effectively. The synergy among these role-playing LLMs results in a cohesive and efficient dialogue generation. Evaluation on MTS-dialogue (Abacha et al., 2023; Ben Abacha et al., 2023), a benchmark dataset for patient-physician dialogues-note pairs, shows that models trained with the augmented synthetic patient-physician dialogues by NoteChat outperforms other state-of-the-art models for generating clinical notes. Our comprehensive automatic and human evaluation demonstrates that NoteChat substantially surpasses state-of-the-art models like ChatGPT and GPT-4 up to 22.78% by domain experts in generating superior synthetic patient-physician dialogues based on clinical notes. NoteChat has the potential to engage patients directly and help clinical documentation, a leading cause of physician burnout (Budd, 2023).","PeriodicalId":521131,"journal":{"name":"Findings of ACL. ACL","volume":"2024 ","pages":"15183-15201"},"PeriodicalIF":0.0,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13248957/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148221653","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Knowledge-Driven Cross-Document Relation Extraction. 知识驱动的跨文档关系抽取。

Findings of ACL. ACL Pub Date : 2024-08-01 DOI: 10.18653/v1/2024.findings-acl.227

Monika Jain, Raghava Mutharaju, Kuldeep Singh, Ramakanth Kavuluru

引用次数: 0

Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models. 知识注入提示：评估和推进临床文本数据生成与大型语言模型。

Findings of ACL. ACL Pub Date : 2024-08-01 DOI: 10.18653/v1/2024.findings-acl.916

Ran Xu, Hejie Cui, Yue Yu, Xuan Kan, Wenqi Shi, Yuchen Zhuang, May D Wang, Wei Jin, Joyce C Ho, Carl Yang

{"title":"Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models.","authors":"Ran Xu, Hejie Cui, Yue Yu, Xuan Kan, Wenqi Shi, Yuchen Zhuang, May D Wang, Wei Jin, Joyce C Ho, Carl Yang","doi":"10.18653/v1/2024.findings-acl.916","DOIUrl":"10.18653/v1/2024.findings-acl.916","url":null,"abstract":"Clinical natural language processing faces challenges like complex medical terminology and clinical contexts. Recently, large language models (LLMs) have shown promise in this domain. Yet, their direct deployment can lead to privacy issues and are constrained by resources. To address this challenge, we delve into synthetic clinical text generation with LLMs for clinical NLP tasks. We propose an innovative, resource-efficient approach, ClinGen, which infuses knowledge into the process. Our model involves clinical knowledge extraction and context-informed LLM prompting. Both clinical topics and writing styles are drawn from external domain-specific knowledge graphs and LLMs to guide data generation. Our extensive empirical study across 8 clinical NLP tasks and 18 datasets reveals that ClinGen consistently enhances performance across various tasks by 7.7%-8.7% on average, effectively aligning the distribution of real datasets and enriching the diversity of generated training instances. Our code is available at https://github.com/ritaranx/ClinGen.","PeriodicalId":521131,"journal":{"name":"Findings of ACL. ACL","volume":"2024 ","pages":"15496-15523"},"PeriodicalIF":0.0,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12949634/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147329129","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0