IEEE Open Journal of the Computer Society最新文献

Sybil-Resilient Publisher Selection Mechanism in Blockchain-Based MCS Systems 基于区块链的MCS系统中的弹性发布者选择机制

IEEE Open Journal of the Computer Society Pub Date : 2025-04-29 DOI: 10.1109/OJCS.2025.3565620

Ankit Agrawal;Ashutosh Bhatia;Kamlesh Tiwari

{"title":"Sybil-Resilient Publisher Selection Mechanism in Blockchain-Based MCS Systems","authors":"Ankit Agrawal;Ashutosh Bhatia;Kamlesh Tiwari","doi":"10.1109/OJCS.2025.3565620","DOIUrl":"https://doi.org/10.1109/OJCS.2025.3565620","url":null,"abstract":"In Blockchain-based Mobile CrowdSensing (BMCS) systems, publishers (data collectors) can exploit the ability to create multiple blockchain identities, enabling Sybil attacks. Selfish, malicious, and collusive Sybil behaviors undermine both reward and majority-based data validation mechanisms, discouraging honest participation and threatening system integrity. Existing solutions often fail to address these issues, particularly in environments dominated by selfish or malicious publishers. This article proposes a novel two-phase publisher selection mechanism to mitigate Sybil attacks in BMCS systems. Phase-I employs a modified Proof-of-Stake (PoS) mechanism with carefully calibrated parameters, including staked amount, coinage, reputation, and randomness. The strategic combination of staked amount and coinage increases the difficulty of Sybil attacks as the system scales over time. Phase-II introduces a lightweight, reputation-based Proof-of-Work (PoW) mechanism tailored for Mobile CrowdSensing (MCS) environments, where puzzle difficulty adjusts dynamically based on the publisher's reputation. Reputation and penalization mechanisms are central to the proposed mechanism, ensuring robust prevention of task domination, selfish behavior, and malicious activities while fostering honest participation. Comprehensive on-chain and off-chain simulations demonstrate the proposed mechanism's effectiveness in mitigating Sybil attacks, reducing their impact, and promoting fair participation.","PeriodicalId":13205,"journal":{"name":"IEEE Open Journal of the Computer Society","volume":"6 ","pages":"586-598"},"PeriodicalIF":0.0,"publicationDate":"2025-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10979902","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143949190","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

SwinSegFormer: Advancing Aerial Image Semantic Segmentation for Flood Detection SwinSegFormer：推进航空图像语义分割洪水检测

IEEE Open Journal of the Computer Society Pub Date : 2025-04-28 DOI: 10.1109/OJCS.2025.3565185

Muhammad Tariq Shaheen;Hafsa Iqbal;Numan Khurshid;Haleema Sadia;Nasir Saeed

{"title":"SwinSegFormer: Advancing Aerial Image Semantic Segmentation for Flood Detection","authors":"Muhammad Tariq Shaheen;Hafsa Iqbal;Numan Khurshid;Haleema Sadia;Nasir Saeed","doi":"10.1109/OJCS.2025.3565185","DOIUrl":"https://doi.org/10.1109/OJCS.2025.3565185","url":null,"abstract":"Semantic segmentation of aerial images is essential for unmanned aerial vehicle (UAV) applications in disaster management, particularly for identifying the flood-affected areas. Traditional techniques face challenges in capturing global semantic information due to their limited receptive fields, and high computational requirement. To address these issues, we propose a novel transformer-based model named SwinSegFormer, which feature a hierarchical encoder that efficiently generates multi-scale high-resolution features along with a lightweight decoder to reduce computational overhead. The proposed model is trained on FloodNet dataset and demonstrates efficient performance on challenging classes such as vehicles, pools, and flooded and non-flooded roads, which are crucial for effective disaster management. Additionally, we developed a post-processing module to categorize areas into flooded and non-flooded. The model achieves a validation mIoU of 75.1%, mDice of 85.4%, and mACC of 87.1%, representing a 10-12% improvement over state-of-the-art vision transformer-based methods. The effectiveness of model is further evaluated on real-world unlabeled flood imagery, highlighting its potential for supporting first aid activities during floods. Relevant codes are available at: <uri>https://github.com/Shaheen1998/SwinSegFormer</uri>.","PeriodicalId":13205,"journal":{"name":"IEEE Open Journal of the Computer Society","volume":"6 ","pages":"645-657"},"PeriodicalIF":0.0,"publicationDate":"2025-04-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10979465","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144090765","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Addressing Security Orchestration Challenges in Next-Generation Networks: A Comprehensive Overview 解决下一代网络中的安全编排挑战：全面概述

IEEE Open Journal of the Computer Society Pub Date : 2025-04-28 DOI: 10.1109/OJCS.2025.3564788

Sadeep Batewela;Pasika Ranaweera;Madhusanka Liyanage;Engin Zeydan;Mika Ylianttila

{"title":"Addressing Security Orchestration Challenges in Next-Generation Networks: A Comprehensive Overview","authors":"Sadeep Batewela;Pasika Ranaweera;Madhusanka Liyanage;Engin Zeydan;Mika Ylianttila","doi":"10.1109/OJCS.2025.3564788","DOIUrl":"https://doi.org/10.1109/OJCS.2025.3564788","url":null,"abstract":"Security Orchestration (SO) plays a pivotal role in ensuring robust, scalable, and efficient management of security mechanisms in next-generation 5G and beyond 5G (B5G) networks. This paper presents a comprehensive analysis of the technical challenges related to Security Orchestration (SO) in these advanced network technologies, focusing on key areas such as network security monitoring, interface standardization, privacy, scalability, multi-domain orchestration, and policy implementation. Additionally, we discuss lessons learned from existing works, identify remaining research gaps, and propose future directions for enhancing SO in 5G and B5G environments. Emerging technologies such as artificial intelligence (AI), blockchain, quantum computing and trusted execution environments (TEE) are also examined for their potential to address these challenges. The paper provides a taxonomy of SO-related issues and offers a roadmap for researchers and practitioners to navigate the evolving landscape of security in 5G and B5G networks.","PeriodicalId":13205,"journal":{"name":"IEEE Open Journal of the Computer Society","volume":"6 ","pages":"669-687"},"PeriodicalIF":0.0,"publicationDate":"2025-04-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10977990","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144108318","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Optimized Multi-Modal Conformer-Based Framework for Continuous Sign Language Recognition 基于多模态一致性的连续手语识别优化框架

IEEE Open Journal of the Computer Society Pub Date : 2025-04-28 DOI: 10.1109/OJCS.2025.3564828

Neena Aloysius;Geetha M;Prema Nedungadi

{"title":"Optimized Multi-Modal Conformer-Based Framework for Continuous Sign Language Recognition","authors":"Neena Aloysius;Geetha M;Prema Nedungadi","doi":"10.1109/OJCS.2025.3564828","DOIUrl":"https://doi.org/10.1109/OJCS.2025.3564828","url":null,"abstract":"This study introduces Efficient ConSignformer, a novel framework advancing Continuous Sign Language Recognition (CSLR) by optimizing the Conformer-based CSLR model, ConSignformer. Central to this advancement is the Sign Query Attention (SQA) module, a computationally efficient self-attention mechanism that enhances both performance and scalability, resulting in the Efficient Conformer. Efficient ConSignformer integrates video embeddings from dual-modal CNN pipelines that process heatmaps and RGB videos, along with temporal learning layers tailored for each modality. These embeddings are further refined using the Efficient Conformer for the fused data from two modalities. To improve recognition accuracy, we employ an innovative task-adaptive supervised pretraining strategy for Efficient Conformer on a curated dataset of continuous Indian Sign Language (ISL). This strategy enables the model to effectively capture intricate data relationships during end-to-end training. Experimental results highlight the significant contributions of the SQA module and the pretraining strategy, with our model achieving competitive performance on benchmark datasets, PHOENIX-2014 and PHOENIX-2014 T. Notably, Efficient ConSignformer excels in recognizing longer sign sequences, leveraging a computationally lightweight Conformer backbone.","PeriodicalId":13205,"journal":{"name":"IEEE Open Journal of the Computer Society","volume":"6 ","pages":"739-749"},"PeriodicalIF":0.0,"publicationDate":"2025-04-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10978102","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144205883","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Large Language Model Enhanced Particle Swarm Optimization for Hyperparameter Tuning for Deep Learning Models 基于大语言模型增强粒子群优化的深度学习模型超参数整定

IEEE Open Journal of the Computer Society Pub Date : 2025-04-25 DOI: 10.1109/OJCS.2025.3564493

Saad Hameed;Basheer Qolomany;Samir Brahim Belhaouari;Mohamed Abdallah;Junaid Qadir;Ala Al-Fuqaha

{"title":"Large Language Model Enhanced Particle Swarm Optimization for Hyperparameter Tuning for Deep Learning Models","authors":"Saad Hameed;Basheer Qolomany;Samir Brahim Belhaouari;Mohamed Abdallah;Junaid Qadir;Ala Al-Fuqaha","doi":"10.1109/OJCS.2025.3564493","DOIUrl":"https://doi.org/10.1109/OJCS.2025.3564493","url":null,"abstract":"Determining the ideal architecture for deep learning models, such as the number of layers and neurons, is a difficult and resource-intensive process that frequently relies on human tuning or computationally costly optimization approaches. While Particle Swarm Optimization (PSO) and Large Language Models (LLMs) have been individually applied in optimization and deep learning, their combined use for enhancing convergence in numerical optimization tasks remains underexplored. Our work addresses this gap by integrating LLMs into PSO to reduce model evaluations and improve convergence for deep learning hyperparameter tuning. The proposed LLM-enhanced PSO method addresses the difficulties of efficiency and convergence by using LLMs (particularly ChatGPT-3.5 and Llama3) to improve PSO performance, allowing for faster achievement of target objectives. Our method speeds up search space exploration by substituting underperforming particle placements with best suggestions offered by LLMs. Comprehensive experiments across three scenarios—(1) optimizing the Rastrigin function, (2) using Long Short-Term Memory (LSTM) networks for time series regression, and (3) using Convolutional Neural Networks (CNNs) for material classification—show that the method significantly improves convergence rates and lowers computational costs. Depending on the application, computational complexity is lowered by 20% to 60% compared to traditional PSO methods. Llama3 achieved a 20% to 40% reduction in model calls for regression tasks, whereas ChatGPT-3.5 reduced model calls by 60% for both regression and classification tasks, all while preserving accuracy and error rates. This groundbreaking methodology offers a very efficient and effective solution for optimizing deep learning models, leading to substantial computational performance improvements across a wide range of applications.","PeriodicalId":13205,"journal":{"name":"IEEE Open Journal of the Computer Society","volume":"6 ","pages":"574-585"},"PeriodicalIF":0.0,"publicationDate":"2025-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10976715","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143929883","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

UNSW-MG24: A Heterogeneous Dataset for Cybersecurity Analysis in Realistic Microgrid Systems UNSW-MG24：现实微电网系统中网络安全分析的异构数据集

IEEE Open Journal of the Computer Society Pub Date : 2025-04-24 DOI: 10.1109/OJCS.2025.3564266

Zhibo Zhang;Benjamin Turnbull;Shabnam Kasra Kermanshahi;Hemanshu Pota;Jiankun Hu

{"title":"UNSW-MG24: A Heterogeneous Dataset for Cybersecurity Analysis in Realistic Microgrid Systems","authors":"Zhibo Zhang;Benjamin Turnbull;Shabnam Kasra Kermanshahi;Hemanshu Pota;Jiankun Hu","doi":"10.1109/OJCS.2025.3564266","DOIUrl":"https://doi.org/10.1109/OJCS.2025.3564266","url":null,"abstract":"One of the major challenges of microgrid systems is the lack of comprehensive Intrusion Detection System (IDS) datasets specifically for realistic microgrid systems' communication. To address the unavailability of comprehensive IDS datasets for realistic microgrid systems, this article presents a UNSW-MG24 dataset based on realistic microgrid testbeds. This dataset contains synthesized benign network traffic from different campus departments, network flow of attack activities, system call traces, and microgrid-specific data from an integrated Festo LabVolt microgrid system. Additionally, pivoting attacks and mimicry attacks are implemented to increase this dataset's heterogeneity for intrusion detection. Comprehensive features such as network flow attributes, system call parameters, and power measurement metrics are extracted from the generated dataset. Finally, a premiminary analysis is presented to elaborate the UNSW-MG24 dataset.This dataset is publicly available for research purposes at UNSW-MG24 dataset.","PeriodicalId":13205,"journal":{"name":"IEEE Open Journal of the Computer Society","volume":"6 ","pages":"543-553"},"PeriodicalIF":0.0,"publicationDate":"2025-04-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10976542","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143929690","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Security Orchestration in 5G and Beyond Smart Network Technologies 5G及超越智能网络技术中的安全编排

IEEE Open Journal of the Computer Society Pub Date : 2025-04-23 DOI: 10.1109/OJCS.2025.3563619

Sadeep Batewela;Madhusanka Liyanage;Engin Zeydan;Mika Ylianttila;Pasika Ranaweera

{"title":"Security Orchestration in 5G and Beyond Smart Network Technologies","authors":"Sadeep Batewela;Madhusanka Liyanage;Engin Zeydan;Mika Ylianttila;Pasika Ranaweera","doi":"10.1109/OJCS.2025.3563619","DOIUrl":"https://doi.org/10.1109/OJCS.2025.3563619","url":null,"abstract":"Security Orchestration (SCO) represents a pivotal element in achieving intelligent and automated security monitoring and management within intricate and dynamic network environments, notably the 5th Generation (5G) and Beyond 5G (B5G) networks. The seamless integration of SCO into emerging 5G/B5G technologies, such as Open Radio Access Networks (ORAN), Zero-touch network and Service Management (ZSM), Software-Defined Networking (SDN), Network Function Virtualization (NFV), Multi-access Edge Computing (MEC), cloud computing, and Network Slicing (NS), is of paramount importance for realizing the vision of zero-touch, self-repair, self-healing, and secure networks. Furthermore, these advanced technologies offer opportunities to enhance SCO's functionalities. This article offers an in-depth exploration of SCO's evolution, significance, functionalities, and critical components, considering its potential within the realm of 5G and B5G network technologies. The primary objective of this specify is to provide an informative analysis of the integration of SCO into 5G and B5G network technologies and how these implementations can be harmoniously combined within a unified framework. Lastly, we present valuable insights, remaining research problems, and future directions toward achieving the goal of zero-touch security and networks.","PeriodicalId":13205,"journal":{"name":"IEEE Open Journal of the Computer Society","volume":"6 ","pages":"554-573"},"PeriodicalIF":0.0,"publicationDate":"2025-04-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10974672","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143929882","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Emerging Computing Tools for Emergency Management: Applications, Limitations and Future Prospects 新兴的应急管理计算工具：应用、限制和未来展望

IEEE Open Journal of the Computer Society Pub Date : 2025-04-23 DOI: 10.1109/OJCS.2025.3563759

Ali Daud;Khameis Mohamed Al Abdouli;Afzal Badshah

{"title":"Emerging Computing Tools for Emergency Management: Applications, Limitations and Future Prospects","authors":"Ali Daud;Khameis Mohamed Al Abdouli;Afzal Badshah","doi":"10.1109/OJCS.2025.3563759","DOIUrl":"https://doi.org/10.1109/OJCS.2025.3563759","url":null,"abstract":"Emerging Technologies (ET) as tools are reshaping every aspect of daily life and revolutionizing the global landscape. Emergency Management (EM), in particular, is a critical domain significantly impacted by these technologies as emergency tools. Organizations and governments actively adopt tools powered by Emerging Technologies (ET) to improve Emergency Management (EM) strategies in a technology driven world. This research aims to comprehensively evaluate the role of ET as tools in EM, focusing on their utilization, effectiveness across different phases, and the challenges associated with their deployment. By exploring key domains influenced by these tools, this research identifies their primary use cases and examines how their integration has transformed the operational framework of EM. This research contributes valuable insights for academia, market entities, and government agencies to integrate ET tools effectively into EM practices, ultimately leading to improved efficiency and a higher probability of saving lives and protecting assets.","PeriodicalId":13205,"journal":{"name":"IEEE Open Journal of the Computer Society","volume":"6 ","pages":"627-644"},"PeriodicalIF":0.0,"publicationDate":"2025-04-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10974680","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144073184","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An Efficient Neural Cell Architecture for Spiking Neural Networks 一种用于脉冲神经网络的高效神经细胞结构

IEEE Open Journal of the Computer Society Pub Date : 2025-04-22 DOI: 10.1109/OJCS.2025.3563423

Kasem Khalil;Ashok Kumar;Magdy Bayoumi

{"title":"An Efficient Neural Cell Architecture for Spiking Neural Networks","authors":"Kasem Khalil;Ashok Kumar;Magdy Bayoumi","doi":"10.1109/OJCS.2025.3563423","DOIUrl":"https://doi.org/10.1109/OJCS.2025.3563423","url":null,"abstract":"Neurons in a Spiking Neural Network (SNN) communicate using electrical pulses or spikes. They fire or trigger conditionally, and learning is sensitive to such triggers' timing and duration. The Leaky Integrate and Fire (LIF) model is the most widely used SNN neuron model. Most existing LIF-based neurons use a fixed spike frequency, which prevents them from attaining near-optimal accuracy. A research challenge is to design energy and area-efficient SNN neural cells that provide high learning accuracy and are scalable. Recently, the idea of tuning the spiking pulses in SNN was proposed and found promising. This work builds on the pulse-tuning idea by proposing an area and energy-efficient, stable, and reconfigurable SNN cell that generates spikes and reconfigures its pulse width to achieve near-optimal learning. It auto-adapts spike rate and duration to attain near-optimal accuracies for various SNN applications. The proposed cell is designed in mixed-signal, known to be beneficial to SNN, implemented using 45-nm technology, occupies an area of 27 <inline-formula><tex-math>$mu {rm m}^{2}$</tex-math></inline-formula>, incurs 1.86 <inline-formula><tex-math>$mu {rm W}$</tex-math></inline-formula>, and yields a high learning performance of 99.12%, 96.37%, and 78.64% in N-MNIST, MNIST, and N-Caltech101 datasets, respectively. The proposed cell attains higher accuracy, scalability, energy, and area economy than the state-of-the-art SNN neurons. Its energy efficiency and compact design make it highly suitable for sensor network applications and embedded systems requiring real-time, low-power neuromorphic computing.","PeriodicalId":13205,"journal":{"name":"IEEE Open Journal of the Computer Society","volume":"6 ","pages":"599-612"},"PeriodicalIF":0.0,"publicationDate":"2025-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10972324","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144073381","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Detailed Comparative Analysis of Automatic Neural Metrics for Machine Translation: BLEURT & BERTScore 用于机器翻译的自动神经度量的详细比较分析：BLEURT和BERTScore

IEEE Open Journal of the Computer Society Pub Date : 2025-04-14 DOI: 10.1109/OJCS.2025.3560333

Aniruddha Mukherjee;Vikas Hassija;Vinay Chamola;Karunesh Kumar Gupta

{"title":"A Detailed Comparative Analysis of Automatic Neural Metrics for Machine Translation: BLEURT & BERTScore","authors":"Aniruddha Mukherjee;Vikas Hassija;Vinay Chamola;Karunesh Kumar Gupta","doi":"10.1109/OJCS.2025.3560333","DOIUrl":"https://doi.org/10.1109/OJCS.2025.3560333","url":null,"abstract":"<sc>Bleurt is a recently introduced metric that employs <sc>Bert, a potent pre-trained language model to assess how well candidate translations compare to a reference translation in the context of machine translation outputs. While traditional metrics like<sc>Bleu rely on lexical similarities, <sc>Bleurt leverages <sc>Bert’s semantic and syntactic capabilities to provide more robust evaluation through complex text representations. However, studies have shown that <sc>Bert, despite its impressive performance in natural language processing tasks can sometimes deviate from human judgment, particularly in specific syntactic and semantic scenarios. Through systematic experimental analysis at the word level, including categorization of errors such as lexical mismatches, untranslated terms, and structural inconsistencies, we investigate how <sc>Bleurt handles various translation challenges. Our study addresses three central questions: What are the strengths and weaknesses of <sc>Bleurt, how do they align with <sc>Bert’s known limitations, and how does it compare with the similar automatic neural metric for machine translation, <sc>BERTScore? Using manually annotated datasets that emphasize different error types and linguistic phenomena, we find that <sc>Bleurt excels at identifying nuanced differences between sentences with high overlap, an area where <sc>BERTScore shows limitations. Our systematic experiments, provide insights for their effective application in machine translation evaluation.","PeriodicalId":13205,"journal":{"name":"IEEE Open Journal of the Computer Society","volume":"6 ","pages":"658-668"},"PeriodicalIF":0.0,"publicationDate":"2025-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10964149","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144178946","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0