Natural Language Processing Journal最新文献

{"title":"Sentiment analysis for stock market research: A bibliometric study","authors":"Xieling Chen ,&nbsp;Haoran Xie ,&nbsp;Zongxi Li ,&nbsp;Han Zhang ,&nbsp;Xiaohui Tao ,&nbsp;Fu Lee Wang","doi":"10.1016/j.nlp.2025.100125","DOIUrl":"10.1016/j.nlp.2025.100125","url":null,"abstract":"<div><div>Sentiment analysis is widely utilized in stock market research. To comprehensively review the field, a bibliometric analysis was performed on 223 articles relating to sentiment analysis for stock markets from 2010 to 2022 collected from Web of Science database. Specifically, we recognized active affiliations, countries/regions, publication sources, and subject areas, identified top cited research articles, visualized scientific collaborations among authors, affiliations, and countries/regions, and revealed main research topics. Findings indicate that computer science journals are active in publishing works on sentiment analysis-facilitated stock market research. The research on sentiment analysis-facilitated stock market has attracted researchers from a wide geographic distribution, who have made significant contributions. The intensity of intra-regional collaborations is higher than that of inter-regional collaborations. Thematic topics regarding stock market research using sentiment analysis were detected using keyword mapping, with the following research topics being widely concerned by scholars: deep learning for stock market prediction, financial news sentiment empowered stock trend forecasting, effects of investor sentiment on financial market, and microblog sentiment classification for market prediction. Findings are helpful in depicting research status to researchers and practitioners, raising their awareness of research frontiers when planning research projects concerning sentiment analysis’s application in stock markets.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"10 ","pages":"Article 100125"},"PeriodicalIF":0.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143139779","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"CapsF: Capsule Fusion for Extracting psychiatric stressors for suicide from Twitter","authors":"Mohammad Ali Dadgostarnia ,&nbsp;Ramin Mousa ,&nbsp;Saba Hesaraki ,&nbsp;Mahdi Hemmasian","doi":"10.1016/j.nlp.2025.100134","DOIUrl":"10.1016/j.nlp.2025.100134","url":null,"abstract":"<div><div>Along with factors such as cancer, blood pressure, street accidents and stroke, suicide has been one of Iran’s main causes of death. One of the main reasons for suicide is psychological stressors. Identifying psychological stressors in an at-risk population can help in the early prevention of suicidal and suicidal behaviours. In recent years, the widespread popularity and flow of real-time information sharing of social media have allowed for potential early intervention in large-scale and even small-scale populations. However, some automated approaches to extract psychiatric stressors from Twitter have been presented, but most of this research has been for non-Persian languages. This study aims to investigate the techniques of detecting psychiatric stress related to suicide from Persian tweets using learning-based methods. The proposed capsule-based approach achieved a binary classification accuracy of 0.83.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"10 ","pages":"Article 100134"},"PeriodicalIF":0.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143454121","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A survey on chatbots and large language models: Testing and evaluation techniques","authors":"Sonali Uttam Singh,&nbsp;Akbar Siami Namin","doi":"10.1016/j.nlp.2025.100128","DOIUrl":"10.1016/j.nlp.2025.100128","url":null,"abstract":"<div><div>Chatbots have been quite developed in the recent decades and evolved along with the field of Artificial Intelligence (AI), enabling powerful capabilities in tasks such as text generation and summarization, sentiment analysis, and many other interesting Natural Language Processing (NLP) based tasks. Advancements in language models (LMs), specifically LLMs, have played an important role in improving the capabilities of chatbots. This survey paper provides a comprehensive overview in chatbot with the integration of LLMs, primarily focusing on the testing, evaluation and performance techniques and frameworks associated with it. The paper discusses the foundational concepts of chatbots and their evolution, highlights the challenges and opportunities they present by reviewing the state-of-the-art papers associated with the chatbots design, testing and evaluation. The survey also delves into the key components of chatbot systems, including Natural Language Understanding (NLU), dialogue management, and Natural Language Generation (NLG), and examine how LLMs have influenced each of these components. Furthermore, the survey examines the ethical considerations and limitations associated with LLMs. The paper primarily focuses on investigating the evaluation techniques and metrics used to assess the performance and effectiveness of these language models. This paper aims to provide an overview of chatbots and highlights the need for an appropriate framework in regards to testing and evaluating these chatbots and the LLMs associated with it in order to provide efficient and proper knowledge to user and potentially improve its quality based on advancements in the field of machine learning.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"10 ","pages":"Article 100128"},"PeriodicalIF":0.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143139778","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evaluation of open and closed-source LLMs for low-resource language with zero-shot, few-shot, and chain-of-thought prompting","authors":"Zabir Al Nazi ,&nbsp;Md. Rajib Hossain ,&nbsp;Faisal Al Mamun","doi":"10.1016/j.nlp.2024.100124","DOIUrl":"10.1016/j.nlp.2024.100124","url":null,"abstract":"<div><div>As the global deployment of Large Language Models (LLMs) increases, the demand for multilingual capabilities becomes more crucial. While many LLMs excel in real-time applications for high-resource languages, few are tailored specifically for low-resource languages. The limited availability of text corpora for low-resource languages, coupled with their minimal utilization during LLM training, hampers the models’ ability to perform effectively in real-time applications. Additionally, evaluations of LLMs are significantly less extensive for low-resource languages. This study offers a comprehensive evaluation of both open-source and closed-source multilingual LLMs focused on low-resource language like Bengali, a language that remains notably underrepresented in computational linguistics. Despite the limited number of pre-trained models exclusively on Bengali, we assess the performance of six prominent LLMs, i.e., three closed-source (GPT-3.5, GPT-4o, Gemini) and three open-source (Aya 101, BLOOM, LLaMA) across key natural language processing (NLP) tasks, including text classification, sentiment analysis, summarization, and question answering. These tasks were evaluated using three prompting techniques: Zero-Shot, Few-Shot, and Chain-of-Thought (CoT). This study found that the default hyperparameters of these pre-trained models, such as temperature, maximum token limit, and the number of few-shot examples, did not yield optimal outcomes and led to hallucination issues in many instances. To address these challenges, ablation studies were conducted on key hyperparameters, particularly temperature and the number of shots, to optimize Few-Shot learning and enhance model performance. The focus of this research is on understanding how these LLMs adapt to low-resource downstream tasks, emphasizing their linguistic flexibility and contextual understanding. Experimental results demonstrated that the closed-source GPT-4o model, utilizing Few-Shot learning and Chain-of-Thought prompting, achieved the highest performance across multiple tasks: an F1 score of 84.54% for text classification, 99.00% for sentiment analysis, a <span><math><mrow><mi>F</mi><msub><mrow><mn>1</mn></mrow><mrow><mi>b</mi><mi>e</mi><mi>r</mi><mi>t</mi></mrow></msub></mrow></math></span> score of 72.87% for summarization, and 58.22% for question answering. For transparency and reproducibility, all methodologies and code from this study are available on our GitHub repository: <span><span>https://github.com/zabir-nabil/bangla-multilingual-llm-eval</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"10 ","pages":"Article 100124"},"PeriodicalIF":0.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143139780","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Tweet question classification for enhancing Tweet Question Answering System","authors":"Chindukuri Mallikarjuna,&nbsp;Sangeetha Sivanesan","doi":"10.1016/j.nlp.2025.100130","DOIUrl":"10.1016/j.nlp.2025.100130","url":null,"abstract":"<div><div>In the evolving landscape of social media, effective Question Answering (QA) systems are crucial for enhancing user engagement and satisfaction. Question classification (QC) is vital for improving the efficiency and accuracy of QA systems. Given the informal and noisy nature of social media texts, which differ significantly from general domain QC datasets, there is a strong need for a specialized tweet QC system for social media QA. In this study, we annotated questions in the Tweet QA dataset, performed tweet question classification, and developed the TweetQC dataset, comprising tweet questions with associated labels. We fine-tuned both general and domain-specific pre-trained language models (PTLMs) on the tweet questions. Experimental results show that TweetRoBERTa achieves the highest F1-score of 91.98, outperforming other PTLMs. Additionally, PTLMs trained on the TREC dataset and evaluated on the TweetQC dataset exhibited an accuracy decline of over 35% compared to models trained and evaluated on the TweetQC dataset. Furthermore, incorporating the expected answer type as an additional feature significantly enhances the performance of tweet QA models. Experimental results proves that TweetRoBERTa achieved the maximum ROUGEL score when compared with existing models for Tweet QA system.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"10 ","pages":"Article 100130"},"PeriodicalIF":0.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143487687","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Semantic-based temporal attention network for Arabic Video Captioning","authors":"Adel Jalal Yousif ,&nbsp;Mohammed H. Al-Jammas","doi":"10.1016/j.nlp.2024.100122","DOIUrl":"10.1016/j.nlp.2024.100122","url":null,"abstract":"<div><div>In recent years, there has been a surge in active research aiming to bridge the gap between computer vision and natural language. In a linguistically diverse region like the Arab world, it is essential to establish a mechanism that facilitates the understanding of visual aspects in native languages. Presents an Arabic video captioning method using an encoder–decoder paradigm based on CNN and LSTM. We employ a temporal attention mechanism along with semantic features to align keyframes with relevant semantic tags. Due to the lack of an Arabic captioning dataset, we use Google’s machine translation system to generate Arabic captions for the MSVD and MSR-VTT datasets, which can be used to train end-to-end Arabic video captioning models. The semantic features are extracted from a neural semantic representation network, which has been specifically trained on Arabic tags for better understanding. Semitic languages like Arabic are heavily attributed to complex morphology, which poses challenges for video captioning. We alleviate these difficulties by employing the AraBERT model as a preprocessing tool. Comprehensive experimental results demonstrate the superior performance of the proposed method compared to state-of-the-art models on two widely-used benchmarks: achieving a CIDEr score of 72.1% on MSVD and 38.0% on MSR-VTT.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"10 ","pages":"Article 100122"},"PeriodicalIF":0.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143140091","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Bibliometric analysis of natural language processing using CiteSpace and VOSviewer","authors":"Xiuming Chen ,&nbsp;Wenjie Tian ,&nbsp;Haoyun Fang","doi":"10.1016/j.nlp.2024.100123","DOIUrl":"10.1016/j.nlp.2024.100123","url":null,"abstract":"<div><div>Natural Language Processing (NLP) holds a pivotal position in the domains of computer science and artificial intelligence (AI). Its focus is on exploring and developing theories and methodologies that facilitate seamless and effective communication between humans and computers through the use of natural language. First of all, In this paper, we employ the bibliometric analysis tools, namely CiteSpace and VOSviewer (Visualization of Similarities viewer) are used as the bibliometric analysis software in this paper to summarize the domain of NLP research and gain insights into its core research priorities. What is more, the Web of Science(WoS) Core Collection database serves as the primary source for data acquisition in this study. The data includes 4803 articles on NLP published from 2011 to May 15, 2024. The trends and types of articles reveal the developmental trajectory and current hotspots in NLP. Finally, the analysis covers eight aspects: volume of published articles, classification, countries, institutional collaboration, author collaboration network, cited author network, co-cited journals, and co-cited references. The applications of NLP are vast, spanning areas such as AI, electronic health records, risk, task analysis, data mining, computational modeling. The findings suggest that the emphasis of future research ought to focus on areas like AI, risk, task analysis, and computational modeling. This paper provides learners and practitioners with a comprehensive insight into the current status and emerging trends in NLP.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"10 ","pages":"Article 100123"},"PeriodicalIF":0.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143140092","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Emotion on the edge: An evaluation of feature representations and machine learning models","authors":"James Thomas Black,&nbsp;Muhammad Zeeshan Shakir","doi":"10.1016/j.nlp.2025.100127","DOIUrl":"10.1016/j.nlp.2025.100127","url":null,"abstract":"<div><div>This paper presents a comprehensive analysis of textual emotion classification, employing a tweet-based dataset to classify emotions such as surprise, love, fear, anger, sadness, and joy. We compare the performances of nine distinct machine learning classification models using Bag of Words (BoW) and Term Frequency-Inverse Document Frequency (TF-IDF) feature representations, as well as a fine-tuned DistilBERT transformer model. We examine the training and inference times of models to determine the most efficient combination when employing an edge architecture, investigating each model’s performance from training to inference using an edge board. The study underscores the significance of combinations of models and features in machine learning, detailing how these choices affect model performance when low computation power needs to be considered. The findings reveal that feature representations significantly influence model efficacy, with BoW and TF-IDF models outperforming DistilBERT. The results show that while BoW models tend to have higher accuracy, the overall performance of TF-IDF models is superior, requiring less time for fitting, Stochastic Gradient Descent and Support Vector Machines proving to be the most efficient in terms of performance and inference times.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"10 ","pages":"Article 100127"},"PeriodicalIF":0.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143139701","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A comparative analysis of encoder only and decoder only models for challenging LLM-generated STEM MCQs using a self-evaluation approach","authors":"Ghada Soliman Ph.D. ,&nbsp;Hozaifa Zaki ,&nbsp;Mohamed Kilany","doi":"10.1016/j.nlp.2025.100131","DOIUrl":"10.1016/j.nlp.2025.100131","url":null,"abstract":"<div><div>Large Language Models (LLMs) have demonstrated impressive capabilities in various tasks, including Multiple-Choice Question Answering (MCQA) evaluated on benchmark datasets with few-shot prompting. Given the absence of benchmark Science, Technology, Engineering, and Mathematics (STEM) datasets on Multiple-Choice Questions (MCQs) created by LLMs, we employed various LLMs (e.g., Vicuna-13B, Bard, and GPT-3.5) to generate MCQs on STEM topics curated from Wikipedia. We evaluated open-source LLM models such as Llama 2-7B and Mistral-7B Instruct, along with an encoder model such as DeBERTa v3 Large, on inference by adding context in addition to fine-tuning with and without context. The results showed that DeBERTa v3 Large and Mistral-7B Instruct outperform Llama 2-7B, highlighting the potential of LLMs with fewer parameters in answering hard MCQs when given the appropriate context through fine-tuning. We also benchmarked the results of these models against closed-source models such as Gemini and GPT-4 on inference with context, showcasing the potential of narrowing the gap between open-source and closed-source models when context is provided. Our work demonstrates the capabilities of LLMs in creating more challenging tasks that can be used as self-evaluation for other models. It also contributes to understanding LLMs’ capabilities in STEM MCQs tasks and emphasizes the importance of context for LLMs with fewer parameters in enhancing their performance.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"10 ","pages":"Article 100131"},"PeriodicalIF":0.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143420235","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Token and part-of-speech fusion for pretraining of transformers with application in automatic cyberbullying detection","authors":"Nor Saiful Azam Bin Nor Azmi ,&nbsp;Michal Ptaszynski ,&nbsp;Fumito Masui ,&nbsp;Juuso Eronen ,&nbsp;Karol Nowakowski","doi":"10.1016/j.nlp.2025.100132","DOIUrl":"10.1016/j.nlp.2025.100132","url":null,"abstract":"<div><div>Cyberbullying detection remains a significant challenge in the context of expanding internet and social media usage. This study proposes a novel pretraining methodology for transformer models, integrating Part-of-Speech (POS) information with a unique way of tokenization. The proposed model, based on the ELECTRA architecture, undergoes pretraining and fine-tuning and is referred to as ELECTRA_POS. By leveraging linguistic structures, this approach improves understanding of context and subtle meaning in the text. Through evaluation using the GLUE benchmark and a dedicated cyberbullying detection dataset, ELECTRA_POS consistently delivers enhanced performance compared to conventional transformer models. Key contributions include the introduction of POS-token fusion techniques and their application to improve cyberbullying detection, as well as insights into how linguistic features influence transformer-based models. The result highlights how integrating POS information into the transformer model improves the detection of harmful online behavior while benefiting other natural language processing (NLP) tasks.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"10 ","pages":"Article 100132"},"PeriodicalIF":0.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143386492","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}