Natural Language Processing Journal最新文献_第7页

BeliN: A novel corpus for Bengali religious news headline generation using contextual feature fusion 基于上下文特征融合的孟加拉语宗教新闻标题生成新语料库

Natural Language Processing Journal Pub Date : 2025-06-01 Epub Date: 2025-03-13 DOI: 10.1016/j.nlp.2025.100138

Md Osama , Ashim Dey , Kawsar Ahmed , Muhammad Ashad Kabir

{"title":"BeliN: A novel corpus for Bengali religious news headline generation using contextual feature fusion","authors":"Md Osama , Ashim Dey , Kawsar Ahmed , Muhammad Ashad Kabir","doi":"10.1016/j.nlp.2025.100138","DOIUrl":"10.1016/j.nlp.2025.100138","url":null,"abstract":"<div><div>Automatic text summarization, particularly headline generation, remains a critical yet under-explored area for Bengali religious news. Existing approaches to headline generation typically rely solely on the article content, overlooking crucial contextual features such as sentiment, category, and aspect. This limitation significantly hinders their effectiveness and overall performance. This study addresses this limitation by introducing a novel corpus, BeliN (Bengali Religious News) – comprising religious news articles from prominent Bangladeshi online newspapers, and <em>MultiGen</em> – a contextual multi-input feature fusion headline generation approach. Leveraging transformer-based pre-trained language models such as BanglaT5, mBART, mT5, and mT0, <em>MultiGen</em> integrates additional contextual features – including category, aspect, and sentiment – with the news content. This fusion enables the model to capture critical contextual information often overlooked by traditional methods. Experimental results demonstrate the superiority of <em>MultiGen</em> over the baseline approach that uses only news content, achieving a BLEU score of 18.61 and ROUGE-L score of 24.19, compared to baseline approach scores of 16.08 and 23.08, respectively. These findings underscore the importance of incorporating contextual features in headline generation for low-resource languages. By bridging linguistic and cultural gaps, this research advances natural language processing for Bengali and other under-represented languages. To promote reproducibility and further exploration, the dataset and implementation code are publicly accessible at <span><span>https://github.com/akabircs/BeliN</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"11 ","pages":"Article 100138"},"PeriodicalIF":0.0,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143629386","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Bag-of-Word approach is not dead: A performance analysis on a myriad of text classification challenges Bag-of-Word方法并没有消亡：对无数文本分类挑战的性能分析

Natural Language Processing Journal Pub Date : 2025-06-01 Epub Date: 2025-05-06 DOI: 10.1016/j.nlp.2025.100154

Mario Graff , Daniela Moctezuma , Eric S. Téllez

{"title":"Bag-of-Word approach is not dead: A performance analysis on a myriad of text classification challenges","authors":"Mario Graff , Daniela Moctezuma , Eric S. Téllez","doi":"10.1016/j.nlp.2025.100154","DOIUrl":"10.1016/j.nlp.2025.100154","url":null,"abstract":"<div><div>The Bag-of-Words (BoW) representation, enhanced with a classifier, was a pioneering approach to solving text classification problems. However, with the advent of transformers and, in general, deep learning architectures, the field has dynamically shifted its focus towards customizing these architectures for various natural language processing tasks, including text classification problems. For a newcomer, it might be impossible to realize that for some text classification problems, the traditional approach is still competitive. This research analyzes the competitiveness of BoW-based representations in different text-classification competitions run in English, Spanish, and Italian. To analyze the performance of these BoW-based representations, we participated in 12 text classification international competitions, summing up 24 tasks comprising five English tasks, seven in Italian, and twelve in Spanish. The results show that the proposed BoW representations have a difference of just 10% w.r.t. the competition winner and less than 2% in three tasks corresponding to author profiling. BoW outperforms BERT solutions and dominates in author profiling tasks.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"11 ","pages":"Article 100154"},"PeriodicalIF":0.0,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143924104","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A transformer based multi task learning approach to multimodal hate speech detection 基于变压器的多任务学习方法用于多模态仇恨语音检测

Natural Language Processing Journal Pub Date : 2025-06-01 Epub Date: 2025-02-20 DOI: 10.1016/j.nlp.2025.100133

Prashant Kapil , Asif Ekbal

{"title":"A transformer based multi task learning approach to multimodal hate speech detection","authors":"Prashant Kapil , Asif Ekbal","doi":"10.1016/j.nlp.2025.100133","DOIUrl":"10.1016/j.nlp.2025.100133","url":null,"abstract":"<div><div>Online hate speech has become a major social issue in recent years, affecting both individuals and society as a whole. Memes are a multimodal kind of internet hate speech that is growing more common. Online memes are often entertaining and harmless. The seemingly innocent meme, on the other hand, transforms into a multimodal form of hate speech—a hateful meme—when specific types of text, graphics, or combinations of both are used. The spread of these harmful or undesirable memes has the potential to disrupt societal peace. Therefore, it is vital to limit inappropriate memes on social media. Multimodal hate speech identification is an inherently difficult and open question. It necessitates collaborative language, visual perception, and multimodal reasoning. This line of research has been progressed in this work by building a multi-task learning-based multimodal system for detecting hateful memes by training four hateful meme data sets concurrently. This MTL framework, which consists of Contrastive Language Image Pretraining (CLIP), UNiversal Image-TExt Representation Learning (UNITER), and BERT, was trained collaboratively to transfer common knowledge while simultaneously training four meme datasets. The results show that the recommended strategy outperforms unimodal and multimodal approaches on four multilingual benchmark datasets, with considerable AUC-ROC, accuracy, and F1-score. The ablation studies are undertaken to emphasise the impact of the sub-component in the MTL model. The confusion matrix is shown as quantitative analysis.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"11 ","pages":"Article 100133"},"PeriodicalIF":0.0,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143760774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Precise length control for large language models 精确长度控制大型语言模型

Natural Language Processing Journal Pub Date : 2025-06-01 Epub Date: 2025-04-01 DOI: 10.1016/j.nlp.2025.100143

Bradley Butcher, Michael O’Keefe, James Titchener

引用次数: 0

It is all in the [MASK]: Simple instruction-tuning enables BERT-like masked language models as generative classifiers 这一切都在[MASK]中：简单的指令调优使bert类屏蔽语言模型成为生成分类器

Natural Language Processing Journal Pub Date : 2025-06-01 Epub Date: 2025-05-06 DOI: 10.1016/j.nlp.2025.100150

Benjamin Clavié, Nathan Cooper, Benjamin Warner

{"title":"It is all in the [MASK]: Simple instruction-tuning enables BERT-like masked language models as generative classifiers","authors":"Benjamin Clavié, Nathan Cooper, Benjamin Warner","doi":"10.1016/j.nlp.2025.100150","DOIUrl":"10.1016/j.nlp.2025.100150","url":null,"abstract":"<div><div>While encoder-only models such as BERT and ModernBERT are ubiquitous in real-world NLP applications, their conventional reliance on task-specific classification heads can limit their applicability compared to decoder-based large language models (LLMs). In this work, we introduce ModernBERT-Large-Instruct, a 0.4B-parameter encoder model that leverages its masked language modeling (MLM) head for generative classification. We design a simple approach, extracting all single-token answers from the FLAN dataset collection, and re-purposing standard MLM pre-training to only mask this single token answer. Our approach employs an intentionally simple training loop and inference mechanism that requires no heavy pre-processing, heavily engineered prompting, or architectural modifications. ModernBERT-Large-Instruct exhibits strong zero-shot performance on both classification and knowledge-based tasks, outperforming similarly sized LLMs on MMLU and achieving 93% of Llama3-1B’s MMLU performance with 60% less parameters. We also demonstrate that, when fine-tuned, the generative approach using the MLM head matches or even surpasses traditional classification-head methods across diverse NLU tasks. This capability emerges specifically in models trained on contemporary, diverse data mixes, with models trained on lower volume, less-diverse data yielding considerably weaker performance. Although preliminary, these results demonstrate the potential of using the original generative masked language modeling head over traditional task-specific heads for downstream tasks. Our work suggests that further exploration into this area is warranted, highlighting many avenues for future improvements.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"11 ","pages":"Article 100150"},"PeriodicalIF":0.0,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143937154","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

LLMs for product classification in e-commerce: A zero-shot comparative study of GPT and claude models 电子商务中产品分类的法学硕士：GPT与claude模型的零射击比较研究

Natural Language Processing Journal Pub Date : 2025-06-01 Epub Date: 2025-03-26 DOI: 10.1016/j.nlp.2025.100142

Konstantinos I. Roumeliotis , Nikolaos D. Tselikas , Dimitrios K. Nasiopoulos

{"title":"LLMs for product classification in e-commerce: A zero-shot comparative study of GPT and claude models","authors":"Konstantinos I. Roumeliotis , Nikolaos D. Tselikas , Dimitrios K. Nasiopoulos","doi":"10.1016/j.nlp.2025.100142","DOIUrl":"10.1016/j.nlp.2025.100142","url":null,"abstract":"<div><div>In the rapidly evolving e-commerce landscape, efficient and accurate product classification is essential for enhancing customer experience and streamlining operations. Traditional product classification methods, which depend heavily on labeled data and manual effort, struggle with scalability and adaptability to diverse product categories. This study explores the transformative potential of large language models (LLMs) for zero-shot product classification in e-commerce, addressing the challenge of automating product categorization without prior labeled training data. We evaluate the performance of four state-of-the-art LLMs — GPT-4o, GPT-4o mini, Claude 3.5 Sonnet, and Claude 3.5 Haiku — on a diverse dataset of 248 product categories, each containing 20 samples, structured into 8 subsets. Each model performs zero-shot classification, assigning products to predefined categories without prior exposure. Our findings reveal significant variations in classification accuracy across models, with certain LLMs demonstrating superior scalability and adaptability for real-world e-commerce applications. Based on these insights, we developed an API software to integrate the top-performing models into e-commerce systems, enhancing automation and efficiency. This study underscores the transformative role of LLMs in revolutionizing e-commerce workflows and recommends their adoption for scalable, intelligent product classification.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"11 ","pages":"Article 100142"},"PeriodicalIF":0.0,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143724168","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Adapting language generation to dialogue environments and users for task-oriented dialogue systems 使语言生成适应对话环境和面向任务的对话系统的用户

Natural Language Processing Journal Pub Date : 2025-06-01 Epub Date: 2025-05-07 DOI: 10.1016/j.nlp.2025.100153

Atsumoto Ohashi, Ryuichiro Higashinaka

{"title":"Adapting language generation to dialogue environments and users for task-oriented dialogue systems","authors":"Atsumoto Ohashi, Ryuichiro Higashinaka","doi":"10.1016/j.nlp.2025.100153","DOIUrl":"10.1016/j.nlp.2025.100153","url":null,"abstract":"<div><div>When a natural language generation (NLG) component is implemented in a real-world task-oriented dialogue system, it is necessary to generate not only natural utterances as learned on training data but also utterances adapted to the dialogue environment (e.g., noise from environmental sounds) and the user (e.g., users with low levels of understanding ability). Inspired by recent advances in reinforcement learning (RL) for language generation tasks, we propose ANTOR, a method for <strong>A</strong>daptive <strong>N</strong>atural language generation for <strong>T</strong>ask-<strong>O</strong>riented dialogue via <strong>R</strong>einforcement learning. In ANTOR, a natural language understanding (NLU) module, which corresponds to the user’s understanding of system utterances, is incorporated into the objective function of RL. If the NLG’s intentions are correctly conveyed to the NLU, the NLG is given a positive reward. We conducted experiments on the two major task-oriented dialogue datasets, MultiWOZ and Schema-Guided Dialogue, and we confirmed that ANTOR could generate adaptive utterances against speech recognition errors and the different vocabulary levels of users. Further analysis revealed that ANTOR adapts to noisy environments and users with different vocabulary levels by prioritizing words that are less likely to cause speech recognition errors and by using words that match the user’s vocabulary level.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"11 ","pages":"Article 100153"},"PeriodicalIF":0.0,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143928216","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Detecting cognitive engagement in online course forums: A review of frameworks and methodologies 检测在线课程论坛中的认知参与：框架和方法的回顾

Natural Language Processing Journal Pub Date : 2025-06-01 Epub Date: 2025-04-11 DOI: 10.1016/j.nlp.2025.100146

Nazmus Sakeef, M. Ali Akber Dewan, Fuhua Lin, Dharamjit Parmar

{"title":"Detecting cognitive engagement in online course forums: A review of frameworks and methodologies","authors":"Nazmus Sakeef, M. Ali Akber Dewan, Fuhua Lin, Dharamjit Parmar","doi":"10.1016/j.nlp.2025.100146","DOIUrl":"10.1016/j.nlp.2025.100146","url":null,"abstract":"<div><div>A key aspect of online learning in higher education involves the utilization of course discussion forums. Assessing the quality of posts, such as cognitive engagement, within online course discussion forums, and determining students’ interest and participation is challenging yet beneficial. This research investigates existing literature on identifying the cognitive engagement of online learners through the analysis of course discussion forums. Essentially, this review examines three educational frameworks - <em>Van Der Meijden’s Knowledge Construction in Synchronous and Asynchronous Discussion Posts (KCSA), Community of Inquiry (CoI), and Interactive, Constructive, Active, and Passive (ICAP)</em>, which have been widely used for students’ cognitive engagement detection analyzing their posts in course discussion forums. This study also examines the natural language processing and deep learning approaches employed and integrated with the above three educational frameworks in the existing literature concerning the detection of cognitive engagement in the context of online learning. The article provides recommendations for enhancing instructional design and fostering student engagement by leveraging cognitive engagement detection. This research underscores the significance of automating the identification of cognitive engagement in online learning and puts forth suggestions for future research directions.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"11 ","pages":"Article 100146"},"PeriodicalIF":0.0,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143848527","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A novel Data Extraction Framework Using Natural Language Processing (DEFNLP) techniques 使用自然语言处理（DEFNLP）技术的新型数据提取框架

Natural Language Processing Journal Pub Date : 2025-06-01 Epub Date: 2025-05-07 DOI: 10.1016/j.nlp.2025.100149

Tayyaba Hussain, Muhammad Usman Akram, Anum Abdul Salam

{"title":"A novel Data Extraction Framework Using Natural Language Processing (DEFNLP) techniques","authors":"Tayyaba Hussain, Muhammad Usman Akram, Anum Abdul Salam","doi":"10.1016/j.nlp.2025.100149","DOIUrl":"10.1016/j.nlp.2025.100149","url":null,"abstract":"<div><div>Evidence through data is critical if government has to address threats faced by the nation, such as pandemics or climate change. Yet several facts about data necessary to inform evidence and science are locked inside publications. We used scientific literature dataset, Coleridge Initiative — Show US the Data, to discover how the data can be used for the public good. In this research, we demonstrate a general Data Extraction Framework Using Natural Language Processing (DEFNLP) Techniques which challenge data scientists to show how publicly funded data has been used to serve science and society. The proposed framework uses NLP libraries and techniques like SpaCy and NER respectively and different huggingface Question Answering (QA) models to predict the datasets used in publications. DEFNLP findings can assist the government in immediate decisions making, accountability, transparent public investments, economic and public health benefits. Until now such an issue having large dataset which belongs to numerous research areas has not been addressed. This approach is domain independent and therefore can be applied to all kind of case studies and scenarios which require data extraction. Our methodology sets the state-of-the-art on Coleridge Initiative dataset, reaching the highest score of 0.554 using salti bert QA model with the less runtime i.e. 417.4 and output of 819 bytes than other QA models e.g., Longformer (runtime: 2710.2, output: 1780 bytes) and BigBird (runtime: 839.4, output: 177020 bytes) with 0.444 and 0.387 score respectively which impressively raised the leaderboard score with an outcome of 0.711. Its computation time to answer each query on CPU is far less i.e. 0.0696s (than 0.3556s and 0.8967s) and has suitable hyperparameters for our dataset as maximum answer length is 64, greater batch size as well as learning rate. In terms of timing and performance, each epoch took around 5 min on average on a computer with output size of 3.27kB which is again far better than other frameworks.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"11 ","pages":"Article 100149"},"PeriodicalIF":0.0,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143942112","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Automatic detection of manipulated Bangla news: A new knowledge-driven approach 自动检测被操纵的孟加拉新闻：一种新的知识驱动的方法

Natural Language Processing Journal Pub Date : 2025-06-01 Epub Date: 2025-05-12 DOI: 10.1016/j.nlp.2025.100155

Aysha Akther, Kazi Masudul Alam, Rameswar Debnath

{"title":"Automatic detection of manipulated Bangla news: A new knowledge-driven approach","authors":"Aysha Akther, Kazi Masudul Alam, Rameswar Debnath","doi":"10.1016/j.nlp.2025.100155","DOIUrl":"10.1016/j.nlp.2025.100155","url":null,"abstract":"<div><div>In recent years, dissemination of misleading news has become easier than ever due to the simplicity of creating and distributing news content on online media platforms. Misleading news detection has become a global topic of interest due to its significant impact on society, economics, and politics. Automatic detection of the veracity of news remains challenging because of its diversity and close resemblance with true events. In many languages, fake news detection has been studied from different perspectives. However, in Bangla, existing endeavors on fake news detection generally relied on linguistic style analysis and latent representation-based machine learning and deep learning models. These models primarily rely on manually labeled annotations. To address these challenges, we proposed a knowledge-based Bangla fake news detection model that does not require model training. In our proposed manipulation detection approach, a news article is automatically labeled as fake or authentic based on an authenticity score that relies on the consistency of knowledge and semantics, underlying sentiment, and credibility of the news source. We also propose a consistent and context-aware manipulated news generation technique to facilitate the detection of partially manipulated Bangla news. We found the proposed model to be a reliable one for the detection of both fake news and partially manipulated news. We also developed a dataset that is balanced according to the number of authentic and fake news for the detection of Bangla fake news, where news items are collected from multiple domains and various news sources. The experimental evaluation of our proposed knowledge-driven approach on the developed dataset has shown 97.08% accuracy for only fake news detection.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"11 ","pages":"Article 100155"},"PeriodicalIF":0.0,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143942113","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0