Natural Language Processing Journal最新文献

{"title":"A hybrid BERT-BiRNN framework for mental health prediction using textual data","authors":"Muhammad Nouman ,&nbsp;Sui Yang Khoo ,&nbsp;M.A. Parvez Mahmud ,&nbsp;Abbas Z. Kouzani","doi":"10.1016/j.nlp.2025.100165","DOIUrl":"10.1016/j.nlp.2025.100165","url":null,"abstract":"<div><div>Effective mental health prediction requires training of artificial intelligence algorithms on relevant datasets obtained from individuals suffering from mental illnesses. This study employs a labelled​ text dataset derived from <em>Lyf Support app</em>. To harness the potential of this dataset for the development of a mental health prediction tool, we propose a novel technique that utilises the bidirectional encoder representations from transformers (BERT) model to identify mental health-related text chats. This technique enables effective and accurate identification of textual content relevant to mental health, facilitating the creation of an advanced prediction model. It is capable of extracting word embeddings retaining the semantic and contextual meaning of words. Then, the bidirectional long-short-term memory (BiLSTM) and bidirectional gated recurrent unit (BiGRU) models are employed as a sequence processing classifier to effectively analyse and detect signs of mental illness from text chats. Extensive experiments are conducted, and the results are compared against the state-of-the-art models, suggesting that our method outperforms the others by achieving 92.4% accuracy. Overall, this study establishes a good foundation for future research endeavours in mental health prediction approaches. The methodologies and findings presented herein pave the way for further advancements and innovations in this field of study.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"12 ","pages":"Article 100165"},"PeriodicalIF":0.0,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144522057","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Detecting indicators of violence in digital text using deep learning","authors":"Abbas Z. Kouzani,&nbsp;Muhammad Nouman","doi":"10.1016/j.nlp.2025.100175","DOIUrl":"10.1016/j.nlp.2025.100175","url":null,"abstract":"<div><div>Individuals who experience violence often use digital platforms to share their experiences and find assistance. Artificial intelligence (AI) techniques have emerged as one of the successful technological strategies used for the detection of indicators of violence in various forms of data, particularly text communications. A hybrid deep learning model is introduced in this paper for the detection of violence indicators in online text communications. It enables the extraction of word embeddings from texts to infer the contextual relationships among words. Additionally, it uses a classifier capable of processing sequential data in both forward as well as backward directions. This approach enables the retention of long-term dependencies from texts while maintaining semantic relationships between words. The word embeddings extraction is implemented with the use of the bidirectional encoder representations from transformer algorithm. The sequence processing classification is implemented by incorporating a combination of parallel layers consisting of the bidirectional long–short-term memory as well as the bidirectional gated recurrent unit algorithms. The developed deep learning architecture is experimentally tested, and the associated results are compared with those of several other machine learning models. The findings are presented and discussed.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"12 ","pages":"Article 100175"},"PeriodicalIF":0.0,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144672206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"RadarTD: A Radar Text Dataset for multi-parameter optimization","authors":"Jackson S. Zaunegger ,&nbsp;Paul G. Singerman ,&nbsp;Ram M. Narayanan ,&nbsp;Muralidhar Rangaswamy","doi":"10.1016/j.nlp.2025.100178","DOIUrl":"10.1016/j.nlp.2025.100178","url":null,"abstract":"<div><div>This paper introduces the radar text dataset (RadarTD) for technical language modeling. This dataset is comprised of sentences containing radar parameters, values, and units determined from published radar literature. Additionally, each statement is assigned a sentiment, goal priority, and goal direction label. In this work, we show how RadarTD may be used to train simple Natural Language Processing (NLP) models to identify the attributes of each sentence listed in RadarTD. Once the NLP models have identified these attributes from text, we can use this information to develop Language Based Cost Functions (LBCF). Our study shows that the proposed text classification model achieves a classification accuracy between 96.7% and 97.8%, while the proposed named entity recognition model achieves an F1 score of 99.7. These findings suggest that the developed models are capable of achieving good performance for both text classification and named entity recognition for autonomous radar applications. We then illustrate an example of how these models could be used with Language Based Cost Functions to develop multi-parameter radar optimization schemes. We also provide a method of providing scalarization weights for each parameter, to improve the results of the optimization process.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"12 ","pages":"Article 100178"},"PeriodicalIF":0.0,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144907138","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Distributed training of large language models: A survey","authors":"Fanlong Zeng ,&nbsp;Wensheng Gan ,&nbsp;Yongheng Wang ,&nbsp;Philip S. Yu","doi":"10.1016/j.nlp.2025.100174","DOIUrl":"10.1016/j.nlp.2025.100174","url":null,"abstract":"<div><div>The emergence of large language models (LLMs) such as ChatGPT has opened up groundbreaking possibilities, enabling a wide range of applications in diverse fields, including healthcare, law, and education. A recent research report highlighted that the performance of these models is often closely tied to their parameter scale, raising a pressing question: how can we effectively train LLMs? This concern is at the forefront of many researchers’ minds. Currently, several distributed training frameworks, such as Megatron-LM and DeepSpeed, are widely used. In this paper, we provide a comprehensive overview of the current state of LLMs, beginning with an introduction to their development status. We then dig into the common parallel strategies employed in LLM distributed training, followed by an examination of the underlying technologies and frameworks that support these models. Next, we discuss the state-of-the-art optimization techniques used in LLMs. Finally, we summarize some key challenges and limitations of current LLM training methods and outline potential future directions for the development of LLMs.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"12 ","pages":"Article 100174"},"PeriodicalIF":0.0,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144714199","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Next-generation image captioning: A survey of methodologies and emerging challenges from transformers to Multimodal Large Language Models","authors":"Huda Diab Abdulgalil,&nbsp;Otman A. Basir","doi":"10.1016/j.nlp.2025.100159","DOIUrl":"10.1016/j.nlp.2025.100159","url":null,"abstract":"<div><div>The widespread availability of visual data on the Internet has fueled a significant interest in image-to-text captioning systems. Automated image captioning remains a challenging multimodal analytics task, integrating advances in both Computer Vision (CV) and Natural Language Processing (NLP) to understand image content and generate semantically meaningful textual descriptions. Modern deep learning-based approaches have supplanted traditional approaches in image captioning, leading to more efficient and sophisticated models. The development of attention mechanisms and transformer-based architectures has further enhanced the modeling of both language and visual data. Despite these gains, challenges such as long-tailed object recognition, bias in training data, and shortcomings in evaluation metrics constrain the capabilities of current models. Furthermore, an important breakthrough has been made with the recent emergence of Multimodal Large Language Models (MLLMs). By incorporating textual and visual data, MLLMs provide improved captioning flexibility, generative capabilities, and reasoning. However, these models introduce new challenges, including faithfulness, grounding, and computational cost. Although relatively few studies have comprehensively surveyed these developments, this paper provides a thorough analysis of Transformer-based captioning approaches, investigates the shift to MLLMs, and discusses associated challenges and opportunities. We also present a performance comparison of the latest models on the MS-COCO benchmark and conclude with perspectives on potential future research directions.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"12 ","pages":"Article 100159"},"PeriodicalIF":0.0,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144313028","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The evolution of language models: From N-Grams to LLMs, and beyond","authors":"Mohammad Ghaseminejad Raeini","doi":"10.1016/j.nlp.2025.100168","DOIUrl":"10.1016/j.nlp.2025.100168","url":null,"abstract":"<div><div>In the last couple of decades language models and artificial intelligence technologies have had significant improvements. Along with computer vision and image processing models, large language models (LLMs) are expected to have big impacts on how AI technologies will evolve. As such, it is important to study how language models have advanced since their inception; and more importantly how they will grow in the future.</div><div>In this article, we provide an overview of the evolution of language models. We start with early statistical and rule-based models. The advancement of language models are discussed all the way to nowadays transformer-based multimodal models (MM-LLMs). We discuss the shortcomings of the current language models and various aspects of the models that need to be improved upon. We also highlight the latest research trends in NLP. Furthermore, we pinpoint important aspects of language models and AI technologies that need further attention. This overview paper provides valuable insights about the progression of language models. It can be motivational and helpful for advancing the state-of-art language models.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"12 ","pages":"Article 100168"},"PeriodicalIF":0.0,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144522058","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Enhancing grammatical documentation for endangered languages with graph-based meaning representation and Loopy Belief Propagation","authors":"Sebastien Christian","doi":"10.1016/j.nlp.2025.100164","DOIUrl":"10.1016/j.nlp.2025.100164","url":null,"abstract":"<div><div>DIG4EL (Digital Inferential Grammars for Endangered Languages) is a method embodied in software designed to assist linguists and teachers in producing grammatical descriptions for endangered languages. DIG4EL integrates linguistic knowledge from extensive databases such as WALS and Grambank with automated observations of controlled data collected using Conversational Questionnaires.</div><div>Linguistic knowledge and automated observations provide priors to a Bayesian network of grammatical parameters, where parameters are interconnected by directional conditional probability matrices derived from statistics on world languages. Inference of unknown parameter values is performed using Loopy Belief Propagation, achieving an average accuracy of 76% and a median accuracy of 85% in an experimental grammatical domain, determining the values of eight parameters related to canonical word order across 116 languages from diverse language families.</div><div>DIG4EL produces outputs either as structured files for computational use, Microsoft Word files, or plain-language grammatical descriptions generated by a Large Language Model. These descriptions rely solely on vetted data and observed examples, with prompts crafted explicitly to prevent external information or hallucinations.</div><div>By leveraging probabilistic modeling and rich, yet quickly assembled linguistic data, DIG4EL provides a powerful, accessible tool for creating grammatical descriptions and language teaching materials with minimal intervention from linguists. It significantly reduces the time and expertise required for traditional documentation workflows, ensuring endangered languages are better documented and taught.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"12 ","pages":"Article 100164"},"PeriodicalIF":0.0,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144329749","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Cross-lingual embedding methods and applications: A systematic review for low-resourced scenarios","authors":"Thapelo Sindane ,&nbsp;Vukosi Marivate ,&nbsp;Abiodun Modupe","doi":"10.1016/j.nlp.2025.100157","DOIUrl":"10.1016/j.nlp.2025.100157","url":null,"abstract":"<div><div>The field of Natural Language Processing (NLP) has achieved significant success in various areas, such as developing large-scale datasets, algorithmic complexity, optimized computing capabilities, refined individual and community expertise, and more, particularly in languages such as English, French, and Spanish. However, such global north unilateral strides have inadvertently created a substantial representation bias towards many languages categorized as low-resourced languages, with the majority being African languages. As a result, rudimentary resources such as stopwords, lemmatizers, stemmers, and word embeddings, as well as advanced multilingual transformer-based models remain under-developed for these languages. Compounding these circumstances is the lack of insights surrounding the development of these resources in the low-resourced context (e.g., how to develop embeddings for morphologically rich languages). Looking back, research priorities aiming to create these resources, largely motivated by the high cost attached to remedying these issues shifted, leading to the rise of alternative methods such as cross-lingual transfer learning (CLTL). CLTL involves transferring domain knowledge gained from supervised training to a domain with limited supervision signals. This study conducts a systematic literature review of CLTL techniques, in the context of cross-lingual models and embeddings, looking at their mathematical foundations, application domains, evaluation metrics, languages covered, and the latest developments. The findings of this study offer valuable insights into the present scenario of CLTL techniques, identifying areas for future research and development to advance cross-lingual natural language processing applications specifically in low-resourced settings.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"12 ","pages":"Article 100157"},"PeriodicalIF":0.0,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144298569","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The fine art of fine-tuning: A structured review of advanced LLM fine-tuning techniques","authors":"Samar Pratap ,&nbsp;Alston Richard Aranha ,&nbsp;Divyanshu Kumar ,&nbsp;Gautam Malhotra ,&nbsp;Anantharaman Palacode Narayana Iyer ,&nbsp;Shylaja S.S.","doi":"10.1016/j.nlp.2025.100144","DOIUrl":"10.1016/j.nlp.2025.100144","url":null,"abstract":"<div><div>Transformer-based models have consistently demonstrated superior accuracy compared to various traditional models across a range of downstream tasks. However, due to their large nature, training or fine-tuning them for specific tasks has heavy computational and memory demands. This causes the creation of specialized transformer-based models to be almost impossible in the generally present constrained scenarios. To tackle this issue and to make these large models more accessible, a plethora of techniques have been developed. In this study, we will be reviewing the types of techniques developed, their impacts and benefits concerning performance and resource usage along with the latest developments in the domain. We have broadly categorized these techniques into six key areas: Changes in Training Method, Changes in Adapter, Quantization, Parameter Selection, Mixture of Experts, and Application based methods. We collated the results of various techniques on common benchmarks and also evaluated their performance on different datasets and base models.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"11 ","pages":"Article 100144"},"PeriodicalIF":0.0,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143738563","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"OdNER: NER resource creation and system development for low-resource Odia language","authors":"Tusarkanta Dalai ,&nbsp;Anupam Das ,&nbsp;Tapas Kumar Mishra ,&nbsp;Pankaj Kumar Sa","doi":"10.1016/j.nlp.2025.100139","DOIUrl":"10.1016/j.nlp.2025.100139","url":null,"abstract":"<div><div>This work aims to enhance the usability of natural language processing (NLP) based systems for the low-resource Odia language by focusing on the development of effective named entity recognition (NER) system. NLP applications rely heavily on NER to extract relevant information from massive amounts of unstructured text. The task of identifying and classifying the named entities included in a given text into a set of predetermined categories is referred to as NER. Already, the NER task has accomplished productive results in English as well as in a number of other European languages. On the other hand, because of a lack of supporting tools and resources, it has not yet been thoroughly investigated in Indian languages, particularly the Odia language. Recently, approaches based on machine learning (ML) and deep learning (DL) have demonstrated exceptional performance when it comes to constructing NLP tasks. Moreover, transformer models, particularly masked-language models (MLM), have demonstrated remarkable efficacy in the NER task; nevertheless, these methods generally call for massive volumes of annotated corpus. Unfortunately, we could not find any open-source NER corpus for the Odia language. The purpose of this research is to compile OdNER, a NER dataset with quality baselines for the low-resource Odia language. The Odia NER corpus OdNER contains 48,000 sentences having 6,71,354 tokens and 98,116 name entities annotated with 12 tags. To establish the quality of our corpus, we use conditional random field (CRF) and BiLSTM model as our baseline models. To demonstrate the efficacy of our dataset, we conduct a comparative evaluation of various transformer-based multilingual language models (IndicBERT, MuRIL, XLM-R) and utilize them to carry out the sequence labeling task for NER. With the pre-trained XLM-R multilingual model, our dataset achieves a maximum F1 score of 90.48%. When it comes to Odia NER, no other work comes close to matching the quality and quantity of ours. We anticipate that, this work will have made substantial progress toward the development of NLP tasks for the Odia language.</div></div>","PeriodicalId":100944,"journal":{"name":"Natural Language Processing Journal","volume":"11 ","pages":"Article 100139"},"PeriodicalIF":0.0,"publicationDate":"2025-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143644136","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}