Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing最新文献

No Free Lunch: Retrieval-Augmented Generation Undermines Fairness in LLMs, Even for Vigilant Users. 没有免费的午餐：检索增强一代破坏了法学硕士的公平性，即使对警惕的用户也是如此。

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing Pub Date : 2025-11-01 DOI: 10.18653/v1/2025.findings-emnlp.984

Mengxuan Hu, Hongyi Wu, Ronghang Zhu, Zihan Guan, Dongliang Guo, Daiqing Qi, Sheng Li

{"title":"No Free Lunch: Retrieval-Augmented Generation Undermines Fairness in LLMs, Even for Vigilant Users.","authors":"Mengxuan Hu, Hongyi Wu, Ronghang Zhu, Zihan Guan, Dongliang Guo, Daiqing Qi, Sheng Li","doi":"10.18653/v1/2025.findings-emnlp.984","DOIUrl":"10.18653/v1/2025.findings-emnlp.984","url":null,"abstract":"Retrieval-Augmented Generation (RAG) is widely adopted for its effectiveness and cost-efficiency in mitigating hallucinations and enhancing the domain-specific generation capabilities of large language models (LLMs). However, is this effectiveness and cost-efficiency truly a free lunch? In this study, we comprehensively investigate the fairness costs associated with RAG by proposing a practical three-level threat model from the perspective of user awareness of fairness. Specifically, varying levels of user fairness awareness result in different degrees of fairness censorship on the external dataset. We examine the fairness implications of RAG using uncensored, partially censored, and fully censored datasets. Our experiments demonstrate that fairness alignment can be easily undermined through RAG without the need for fine-tuning or retraining. Even with fully censored and supposedly unbiased external datasets, RAG can lead to biased outputs. Our findings underscore the limitations of current alignment methods in the context of RAG-based LLMs and highlight the urgent need for new strategies to ensure fairness. We propose potential mitigations and call for further research to develop robust fairness safeguards in RAG-based LLMs.","PeriodicalId":521132,"journal":{"name":"Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing","volume":"2025 ","pages":"18145-18170"},"PeriodicalIF":0.0,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12713515/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145807345","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Visual Self-Refinement for Autoregressive Models. 自回归模型的视觉自细化。

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing Pub Date : 2025-11-01 DOI: 10.18653/v1/2025.findings-emnlp.1161

Jiamian Wang, Ziqi Zhou, Chaithanya Kumar Mummadi, Sohail Dianat, Majid Rabbani, Raghuveer Rao, Chen Qiu, Zhiqiang Tao

引用次数: 0

Large Language Models with Temporal Reasoning for Longitudinal Clinical Summarization and Prediction. 具有时间推理的大型语言模型用于纵向临床总结和预测。

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing Pub Date : 2025-11-01 DOI: 10.18653/v1/2025.findings-emnlp.1128

Maya Kruse, Shiyue Hu, Nicholas Derby, Yifu Wu, Samantha Stonbraker, Bingsheng Yao, Dakuo Wang, Elizabeth Goldberg, Yanjun Gao

{"title":"Large Language Models with Temporal Reasoning for Longitudinal Clinical Summarization and Prediction.","authors":"Maya Kruse, Shiyue Hu, Nicholas Derby, Yifu Wu, Samantha Stonbraker, Bingsheng Yao, Dakuo Wang, Elizabeth Goldberg, Yanjun Gao","doi":"10.18653/v1/2025.findings-emnlp.1128","DOIUrl":"10.18653/v1/2025.findings-emnlp.1128","url":null,"abstract":"Recent advances in large language models (LLMs) have shown potential in clinical text summarization, but their ability to handle long patient trajectories with multi-modal data spread across time remains underexplored. This study systematically evaluates several state-of-the-art open-source LLMs, their Retrieval Augmented Generation (RAG) variants and chain-of-thought (CoT) prompting on long-context clinical summarization and prediction. We examine their ability to synthesize structured and unstructured Electronic Health Records (EHR) data while reasoning over temporal coherence, by re-engineering existing tasks, including discharge summarization and diagnosis prediction from two publicly available EHR datasets. Our results indicate that long context windows improve input integration but do not consistently enhance clinical reasoning, and LLMs are still struggling with temporal progression and rare disease prediction. While RAG shows improvements in hallucination in some cases, it does not fully address these limitations. Our work fills the gap in long clinical text summarization, establishing a foundation for evaluating LLMs with multi-modal data and temporal reasoning.","PeriodicalId":521132,"journal":{"name":"Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing","volume":"2025 ","pages":"20715-20735"},"PeriodicalIF":0.0,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12702291/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145764953","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MedCOD: Enhancing English-to-Spanish Medical Translation of Large Language Models Using Enriched Chain-of-Dictionary Framework. MedCOD：利用丰富的词典链框架增强大型语言模型的英西医学翻译。

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing Pub Date : 2025-11-01 DOI: 10.18653/v1/2025.findings-emnlp.350

Md Shahidul Salim, Lian Fu, Arav Adikesh Ramakrishnan, Zonghai Yao, Hong Yu

{"title":"MedCOD: Enhancing English-to-Spanish Medical Translation of Large Language Models Using Enriched Chain-of-Dictionary Framework.","authors":"Md Shahidul Salim, Lian Fu, Arav Adikesh Ramakrishnan, Zonghai Yao, Hong Yu","doi":"10.18653/v1/2025.findings-emnlp.350","DOIUrl":"10.18653/v1/2025.findings-emnlp.350","url":null,"abstract":"We present MedCOD (Medical Chain-of-Dictionary), a hybrid framework designed to improve English-to-Spanish medical translation by integrating domain-specific structured knowledge into large language models (LLMs). MedCOD integrates domain-specific knowledge from both the Unified Medical Language System (UMLS) and the LLM-as-Knowledge-Base (LLM-KB) paradigm to enhance structured prompting and fine-tuning. We constructed a parallel corpus of 2,999 English-Spanish MedlinePlus articles and a 100-sentence test set annotated with structured medical contexts. Four open-source LLMs (Phi-4, Qwen2.5-14B, Qwen2.5-7B, and LLaMA-3.1-8B) were evaluated using structured prompts that incorporated multilingual variants, medical synonyms, and UMLS-derived definitions, combined with LoRA-based fine-tuning. Experimental results demonstrate that MedCOD significantly improves translation quality across all models. For example, Phi-4 with MedCOD and fine-tuning achieved BLEU 44.23, chrF++ 28.91, and COMET 0.863, surpassing strong baseline models like GPT-4o and GPT-4o-mini. Ablation studies confirm that both MedCOD prompting and model adaptation independently contribute to performance gains, with their combination yielding the highest improvements. These findings highlight the potential of structured knowledge integration to enhance LLMs for medical translation tasks.","PeriodicalId":521132,"journal":{"name":"Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing","volume":"2025 ","pages":"6579-6597"},"PeriodicalIF":0.0,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12878947/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146145860","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Using tournaments to calculate AUROC for zero-shot classification with LLMs. 使用锦标赛计算带有llm的零杆分类的AUROC。

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing Pub Date : 2025-11-01 DOI: 10.18653/v1/2025.findings-emnlp.1281

WonJin Yoon, Ian Bulovic, Timothy A Miller

引用次数: 0

Large Language Models for Controllable Multi-property Multi-objective Molecule Optimization. 可控多属性多目标分子优化的大语言模型。

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing Pub Date : 2025-11-01 DOI: 10.18653/v1/2025.findings-emnlp.1145

Vishal Dey, Xiao Hu, Xia Ning

{"title":"Large Language Models for Controllable Multi-property Multi-objective Molecule Optimization.","authors":"Vishal Dey, Xiao Hu, Xia Ning","doi":"10.18653/v1/2025.findings-emnlp.1145","DOIUrl":"10.18653/v1/2025.findings-emnlp.1145","url":null,"abstract":"In real-world drug design, molecule optimization requires selectively improving multiple molecular properties up to pharmaceutically relevant levels, while maintaining others that already meet such criteria. However, existing computational approaches and instruction-tuned LLMs fail to capture such nuanced property-specific objectives, limiting their practical applicability. To address this, we introduce C-MuMOInstruct, the first instruction-tuning dataset focused on multi-property optimization with explicit, property-specific objectives. Leveraging C-MuMOInstruct, we develop GeLLM4O-Cs, a series of instruction-tuned LLMs that can perform targeted property-specific optimization. Our experiments across 5 in-distribution and 5 out-of-distribution tasks show that GeLLM4O-Cs consistently outperform strong baselines, achieving up to 126% higher success rate. Notably, GeLLM4O-Cs exhibit impressive 0-shot generalization to novel optimization tasks and unseen instructions. This offers a step toward a foundational LLM to support realistic, diverse optimizations with property-specific objectives. C-MuMOInstruct and code are accessible through https://github.com/ninglab/GeLLMO-C.","PeriodicalId":521132,"journal":{"name":"Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing","volume":"2025 ","pages":"20996-21023"},"PeriodicalIF":0.0,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12765489/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145907429","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Unequal Scientific Recognition in the Age of LLMs. 法学硕士时代的不平等科学认知。

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing Pub Date : 2025-11-01 DOI: 10.18653/v1/2025.findings-emnlp.1279

Yixuan Liu, Ábel Elekes, Jianglin Lu, Rodrigo Dorantes-Gilardi, Albert-László Barabási

{"title":"Unequal Scientific Recognition in the Age of LLMs.","authors":"Yixuan Liu, Ábel Elekes, Jianglin Lu, Rodrigo Dorantes-Gilardi, Albert-László Barabási","doi":"10.18653/v1/2025.findings-emnlp.1279","DOIUrl":"10.18653/v1/2025.findings-emnlp.1279","url":null,"abstract":"Large language models (LLMs) are reshaping how scientific knowledge is accessed and represented. This study evaluates the extent to which popular and frontier LLMs including GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro recognize scientists, benchmarking their outputs against OpenAlex and Wikipedia. Using a dataset focusing on 100,000 physicists from OpenAlex to evaluate LLM recognition, we uncover substantial disparities: LLMs exhibit selective and inconsistent recognition patterns. Recognition correlates strongly with scholarly impact such as citations, and remains uneven across gender and geography. Women researchers, and researchers from Africa, Asia, and Latin America are significantly underrecognized. We further examine the role of training data provenance, identifying Wikipedia as a potential sources that contributes to recognition gaps. Our findings highlight how LLMs can reflect, and potentially amplify existing disparities in science, underscoring the need for more transparent and inclusive knowledge systems.","PeriodicalId":521132,"journal":{"name":"Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing","volume":"2025 ","pages":"23558-23568"},"PeriodicalIF":0.0,"publicationDate":"2025-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC13403178/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148612445","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multi-label Sequential Sentence Classification via Large Language Model. 基于大语言模型的多标签顺序句子分类。

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing Pub Date : 2024-11-01 DOI: 10.18653/v1/2024.findings-emnlp.944

Mengfei Lan, Lecheng Zheng, Shufan Ming, Halil Kilicoglu

{"title":"Multi-label Sequential Sentence Classification via Large Language Model.","authors":"Mengfei Lan, Lecheng Zheng, Shufan Ming, Halil Kilicoglu","doi":"10.18653/v1/2024.findings-emnlp.944","DOIUrl":"10.18653/v1/2024.findings-emnlp.944","url":null,"abstract":"Sequential sentence classification (SSC) in scientific publications is crucial for supporting downstream tasks such as fine-grained information retrieval and extractive summarization. However, current SSC methods are constrained by model size, sequence length, and single-label setting. To address these limitations, this paper proposes LLM-SSC, a large language model (LLM)-based framework for both single- and multi-label SSC tasks. Unlike previous approaches that employ small- or medium-sized language models, the proposed framework utilizes LLMs to generate SSC labels through designed prompts, which enhance task understanding by incorporating demonstrations and a query to describe the prediction target. We also present a multi-label contrastive learning loss with auto-weighting scheme, enabling the multi-label classification task. To support our multi-label SSC analysis, we introduce and release a new dataset, BIORC800, which mainly contains unstructured abstracts in the biomedical domain with manual annotations. Experiments demonstrate LLM-SSC's strong performance in SSC under both in-context learning and task-specific tuning settings. We release BIORC800 and our code at: https://github.com/ScienceNLPLab/LLM-SSC.","PeriodicalId":521132,"journal":{"name":"Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing","volume":"2024 ","pages":"16086-16104"},"PeriodicalIF":0.0,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12758481/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145902255","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks. 在生物医学基准中，语言模型对药物名称出奇地脆弱。

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing Pub Date : 2024-11-01 DOI: 10.18653/v1/2024.findings-emnlp.726

Jack Gallifant, Shan Chen, Pedro Moreira, Nikolaj Munch, Mingye Gao, Jackson Pond, Hugo Aerts, Leo Anthony Celi, Thomas Hartvigsen, Danielle S Bitterman

引用次数: 0

DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer's Disease Questions with Scientific Literature. DALK: LLMs和KG的动态共同增强，用科学文献回答阿尔茨海默病问题。

Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing Pub Date : 2024-11-01 DOI: 10.18653/v1/2024.findings-emnlp.119

Dawei Li, Shu Yang, Zhen Tan, Jae Young Baik, Sukwon Yun, Joseph Lee, Aaron Chacko, Bojian Hou, Duy Duong-Tran, Ying Ding, Huan Liu, Li Shen, Tianlong Chen

{"title":"DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer's Disease Questions with Scientific Literature.","authors":"Dawei Li, Shu Yang, Zhen Tan, Jae Young Baik, Sukwon Yun, Joseph Lee, Aaron Chacko, Bojian Hou, Duy Duong-Tran, Ying Ding, Huan Liu, Li Shen, Tianlong Chen","doi":"10.18653/v1/2024.findings-emnlp.119","DOIUrl":"10.18653/v1/2024.findings-emnlp.119","url":null,"abstract":"Recent advancements in large language models (LLMs) have achieved promising performances across various applications. Nonetheless, the ongoing challenge of integrating long-tail knowledge continues to impede the seamless adoption of LLMs in specialized domains. In this work, we introduce DALK, a.k.a. Dynamic Co-Augmentation of LLMs and KG, to address this limitation and demonstrate its ability on studying Alzheimer's Disease (AD), a specialized sub-field in biomedicine and a global health priority. With a synergized framework of LLM and KG mutually enhancing each other, we first leverage LLM to construct an evolving AD-specific knowledge graph (KG) sourced from AD-related scientific literature, and then we utilize a coarse-to-fine sampling method with a novel self-aware knowledge retrieval approach to select appropriate knowledge from the KG to augment LLM inference capabilities. The experimental results, conducted on our constructed AD question answering (ADQA) benchmark, underscore the efficacy of DALK. Additionally, we perform a series of detailed analyses that can offer valuable insights and guidelines for the emerging topic of mutually enhancing KG and LLM.","PeriodicalId":521132,"journal":{"name":"Findings of ACL. EMNLP. Conference on Empirical Methods in Natural Language Processing","volume":"2024 ","pages":"2187-2205"},"PeriodicalIF":0.0,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12720203/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145822693","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0