Proceedings of the conference. Association for Computational Linguistics. Meeting最新文献

Medical Vision-Language Pre-Training for Brain Abnormalities. 针对大脑异常的医学视觉语言预培训。

Proceedings of the conference. Association for Computational Linguistics. Meeting Pub Date : 2024-05-01

Masoud Monajatipoor, Zi-Yi Dou, Aichi Chien, Nanyun Peng, Kai-Wei Chang

{"title":"Medical Vision-Language Pre-Training for Brain Abnormalities.","authors":"Masoud Monajatipoor, Zi-Yi Dou, Aichi Chien, Nanyun Peng, Kai-Wei Chang","doi":"","DOIUrl":"","url":null,"abstract":"Vision-language models have become increasingly powerful for tasks that require an understanding of both visual and linguistic elements, bridging the gap between these modalities. In the context of multimodal clinical AI, there is a growing need for models that possess domain-specific knowledge, as existing models often lack the expertise required for medical applications. In this paper, we take brain abnormalities as an example to demonstrate how to automatically collect medical image-text aligned data for pretraining from public resources such as PubMed. In particular, we present a pipeline that streamlines the pre-training process by initially collecting a large brain image-text dataset from case reports and published journals and subsequently constructing a high-performance vision-language model tailored to specific medical tasks. We also investigate the unique challenge of mapping subfigures to subcaptions in the medical domain. We evaluated the resulting model with quantitative and qualitative intrinsic evaluations. The resulting dataset and our code can be found here https://github.com/masoud-monajati/MedVL_pretraining_pipeline.","PeriodicalId":74541,"journal":{"name":"Proceedings of the conference. Association for Computational Linguistics. Meeting","volume":"2024 LREC/COLING","pages":"11159-11164"},"PeriodicalIF":0.0,"publicationDate":"2024-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11238846/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141617775","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

HiGen: Hierarchy-Aware Sequence Generation for Hierarchical Text Classification.

Proceedings of the conference. Association for Computational Linguistics. Meeting Pub Date : 2024-03-01

Vidit Jain, Mukund Rungta, Yuchen Zhuang, Yue Yu, Zeyu Wang, Mu Gao, Jeffrey Skolnick, Chao Zhang

{"title":"HiGen: Hierarchy-Aware Sequence Generation for Hierarchical Text Classification.","authors":"Vidit Jain, Mukund Rungta, Yuchen Zhuang, Yue Yu, Zeyu Wang, Mu Gao, Jeffrey Skolnick, Chao Zhang","doi":"","DOIUrl":"","url":null,"abstract":"Hierarchical text classification (HTC) is a complex subtask under multi-label text classification, characterized by a hierarchical label taxonomy and data imbalance. The best-performing models aim to learn a static representation by combining document and hierarchical label information. However, the relevance of document sections can vary based on the hierarchy level, necessitating a dynamic document representation. To address this, we propose HiGen, a text-generation-based framework utilizing language models to encode dynamic text representations. We introduce a level-guided loss function to capture the relationship between text and label name semantics. Our approach incorporates a task-specific pretraining strategy, adapting the language model to in-domain knowledge and significantly enhancing performance for classes with limited examples. Furthermore, we present a new and valuable dataset called ENZYME, designed for HTC, which comprises articles from PubMed with the goal of predicting Enzyme Commission (EC) numbers. Through extensive experiments on the ENZYME dataset and the widely recognized WOS and NYT datasets, our methodology demonstrates superior performance, surpassing existing approaches while efficiently handling data and mitigating class imbalance. We release our code and dataset here: https://github.com/viditjain99/HiGen.","PeriodicalId":74541,"journal":{"name":"Proceedings of the conference. Association for Computational Linguistics. Meeting","volume":"2024 EACL","pages":"1354-1368"},"PeriodicalIF":0.0,"publicationDate":"2024-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11781299/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143070252","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Improving the Transferability of Clinical Note Section Classification Models with BERT and Large Language Model Ensembles. 利用BERT和大型语言模型集成提高临床笔记部分分类模型的可移植性。

Proceedings of the conference. Association for Computational Linguistics. Meeting Pub Date : 2023-07-01

Weipeng Zhou, Dmitriy Dligach, Majid Afshar, Yanjun Gao, Timothy A Miller

引用次数: 0

Improving the Transferability of Clinical Note Section Classification Models with BERT and Large Language Model Ensembles 利用BERT和大型语言模型集成提高临床笔记部分分类模型的可移植性

Proceedings of the conference. Association for Computational Linguistics. Meeting Pub Date : 2023-07-01 DOI: 10.18653/v1/2023.clinicalnlp-1.16

Weipeng Zhou, M. Afshar, Dmitriy Dligach, Yanjun Gao, Timothy Miller

引用次数: 0

Less Likely Brainstorming: Using Language Models to Generate Alternative Hypotheses. 不太可能的头脑风暴:使用语言模型生成替代假设。

Proceedings of the conference. Association for Computational Linguistics. Meeting Pub Date : 2023-07-01 DOI: 10.18653/v1/2023.findings-acl.794

Liyan Tang, Yifan Peng, Yanshan Wang, Ying Ding, Greg Durrett, Justin F Rousseau

{"title":"Less Likely Brainstorming: Using Language Models to Generate Alternative Hypotheses.","authors":"Liyan Tang, Yifan Peng, Yanshan Wang, Ying Ding, Greg Durrett, Justin F Rousseau","doi":"10.18653/v1/2023.findings-acl.794","DOIUrl":"https://doi.org/10.18653/v1/2023.findings-acl.794","url":null,"abstract":"A human decision-maker benefits the most from an AI assistant that corrects for their biases. For problems such as generating interpretation of a radiology report given findings, a system predicting only highly likely outcomes may be less useful, where such outcomes are already obvious to the user. To alleviate biases in human decision-making, it is worth considering a broad differential diagnosis, going beyond the most likely options. We introduce a new task, \"less likely brainstorming,\" that asks a model to generate outputs that humans think are relevant but less likely to happen. We explore the task in two settings: a brain MRI interpretation generation setting and an everyday commonsense reasoning setting. We found that a baseline approach of training with less likely hypotheses as targets generates outputs that humans evaluate as either likely or irrelevant nearly half of the time; standard MLE training is not effective. To tackle this problem, we propose a controlled text generation method that uses a novel contrastive learning strategy to encourage models to differentiate between generating likely and less likely outputs according to humans. We compare our method with several state-of-the-art controlled text generation models via automatic and human evaluations and show that our models' capability of generating less likely outputs is improved.","PeriodicalId":74541,"journal":{"name":"Proceedings of the conference. Association for Computational Linguistics. Meeting","volume":"2023 ","pages":"12532-12555"},"PeriodicalIF":0.0,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10494958/pdf/nihms-1923571.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10263511","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Revisiting Relation Extraction in the era of Large Language Models. 再论大语言模型时代的关系抽取。

Proceedings of the conference. Association for Computational Linguistics. Meeting Pub Date : 2023-07-01 DOI: 10.18653/v1/2023.acl-long.868

Somin Wadhwa, Silvio Amir, Byron C Wallace

{"title":"Revisiting Relation Extraction in the era of Large Language Models.","authors":"Somin Wadhwa, Silvio Amir, Byron C Wallace","doi":"10.18653/v1/2023.acl-long.868","DOIUrl":"https://doi.org/10.18653/v1/2023.acl-long.868","url":null,"abstract":"Relation extraction (RE) is the core NLP task of inferring semantic relationships between entities from text. Standard supervised RE techniques entail training modules to tag tokens comprising entity spans and then predict the relationship between them. Recent work has instead treated the problem as a sequence-to-sequence task, linearizing relations between entities as target strings to be generated conditioned on the input. Here we push the limits of this approach, using larger language models (GPT-3 and Flan-T5 large) than considered in prior work and evaluating their performance on standard RE tasks under varying levels of supervision. We address issues inherent to evaluating generative approaches to RE by doing human evaluations, in lieu of relying on exact matching. Under this refined evaluation, we find that: (1) Few-shot prompting with GPT-3 achieves near SOTA performance, i.e., roughly equivalent to existing fully supervised models; (2) Flan-T5 is not as capable in the few-shot setting, but supervising and fine-tuning it with Chain-of-Thought (CoT) style explanations (generated via GPT-3) yields SOTA results. We release this model as a new baseline for RE tasks.","PeriodicalId":74541,"journal":{"name":"Proceedings of the conference. Association for Computational Linguistics. Meeting","volume":"2023 ","pages":"15566-15589"},"PeriodicalIF":0.0,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10482322/pdf/nihms-1912166.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10181357","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Overview of the Problem List Summarization (ProbSum) 2023 Shared Task on Summarizing Patients' Active Diagnoses and Problems from Electronic Health Record Progress Notes. 问题列表总结概述(ProbSum) 2023关于总结患者主动诊断和电子健康记录进度记录问题的共享任务。

Proceedings of the conference. Association for Computational Linguistics. Meeting Pub Date : 2023-07-01 DOI: 10.18653/v1/2023.bionlp-1.43

Yanjun Gao, Dmitriy Dligach, Timothy Miller, Matthew M Churpek, Majid Afshar

{"title":"Overview of the Problem List Summarization (ProbSum) 2023 Shared Task on Summarizing Patients' Active Diagnoses and Problems from Electronic Health Record Progress Notes.","authors":"Yanjun Gao, Dmitriy Dligach, Timothy Miller, Matthew M Churpek, Majid Afshar","doi":"10.18653/v1/2023.bionlp-1.43","DOIUrl":"https://doi.org/10.18653/v1/2023.bionlp-1.43","url":null,"abstract":"The BioNLP Workshop 2023 initiated the launch of a shared task on Problem List Summarization (ProbSum) in January 2023. The aim of this shared task is to attract future research efforts in building NLP models for real-world diagnostic decision support applications, where a system generating relevant and accurate diagnoses will augment the healthcare providers' decision-making process and improve the quality of care for patients. The goal for participants is to develop models that generated a list of diagnoses and problems using input from the daily care notes collected from the hospitalization of critically ill patients. Eight teams submitted their final systems to the shared task leaderboard. In this paper, we describe the tasks, datasets, evaluation metrics, and baseline systems. Additionally, the techniques and results of the evaluation of the different approaches tried by the participating teams are summarized.","PeriodicalId":74541,"journal":{"name":"Proceedings of the conference. Association for Computational Linguistics. Meeting","volume":"2023 ","pages":"461-467"},"PeriodicalIF":0.0,"publicationDate":"2023-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10426335/pdf/nihms-1923203.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"10017111","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

End-to-end clinical temporal information extraction with multi-head attention. 使用多头注意力进行端到端临床时间信息提取。

Proceedings of the conference. Association for Computational Linguistics. Meeting Pub Date : 2023-07-01

Timothy Miller, Steven Bethard, Dmitriy Dligach, Guergana Savova

引用次数: 0

Multi-Task Training with In-Domain Language Models for Diagnostic Reasoning. 基于领域内语言模型的诊断推理多任务训练。

Proceedings of the conference. Association for Computational Linguistics. Meeting Pub Date : 2023-07-01

Brihat Sharma, Yanjun Gao, Timothy Miller, Matthew M Churpek, Majid Afshar, Dmitriy Dligach

引用次数: 0

End-to-end clinical temporal information extraction with multi-head attention 基于多头注意力的端到端临床时间信息提取

Proceedings of the conference. Association for Computational Linguistics. Meeting Pub Date : 2023-07-01 DOI: 10.18653/v1/2023.bionlp-1.28

Timothy Miller, S. Bethard, Dmitriy Dligach, G. Savova

引用次数: 0