Special issue on trustworthy and safe AI: Interpretability, robustness, and security

IF 2 4区 计算机科学 Q3 ENGINEERING, ELECTRICAL & ELECTRONIC
ETRI Journal Pub Date : 2026-10-20 Epub Date: 2026-10-15 DOI:10.4218/etr2.70162
Sungwon Yi, Dae Hun Nyang, Jaehyung Kim, Jaegul Choo, Daeseon Choi, Soonmin Hwang
{"title":"Special issue on trustworthy and safe AI: Interpretability, robustness, and security","authors":"Sungwon Yi,&nbsp;Dae Hun Nyang,&nbsp;Jaehyung Kim,&nbsp;Jaegul Choo,&nbsp;Daeseon Choi,&nbsp;Soonmin Hwang","doi":"10.4218/etr2.70162","DOIUrl":null,"url":null,"abstract":"<p>Recent advances in artificial intelligence (AI), particularly in large language models (LLMs) and physical robotics, have rapidly reshaped daily life and industrial applications. As these systems become more capable and autonomous, ensuring their safety, trustworthiness, robustness, and alignment with human values has become increasingly important. In response to these challenges, this special issue on “Trustworthy and Safe AI: Interpretability, Robustness, and Security” brings together recent research that advances the safety and trustworthiness of AI across diverse models, systems, and real-world applications.</p><p>This special issue received 50 submissions, reflecting the growing research interest in trustworthy and safe AI. Following a rigorous peer-review process, 11 papers were selected for publication. Together, these studies span several interconnected dimensions of the field, ranging from emerging security threats in LLMs and multi-agent systems to robustness, autonomous system safety, and explainability for practical deployment.</p><p>The following five papers focus on emerging security and safety threats across modern AI systems. These studies examine how vulnerabilities arise from adversarial attacks, multi-agent interactions, social context, multilingual settings, and hidden information while exploring the corresponding mechanisms for detection and defense.</p><p>In [<span>1</span>], the authors investigate a more evasive mixed clean/dirty-label X-to-X backdoor attack and propose an ensemble defense that combines complementary detectors such as Neural Cleanse, PT-RED, CEPA, and MMBD. This work is significant because it addresses both robustness and interpretability, and the ensemble improves resilience across diverse backdoor mechanisms while also supporting trigger inversion to help explain the nature of the detected attack. By jointly considering stronger adversarial attacks, generalized defense, and interpretable backdoor analysis, this study aligns well with the themes of Trustworthy and Safe AI: Interpretability, Robustness, and Security.</p><p>In [<span>2</span>], the authors study how a single malicious agent can compromise the collective decision-making of otherwise well-aligned LLM agents in a collaborative financial multi-agent environment and proposes a data-lineage-based systemic defense. Its main contribution is to show that the interaction dynamics themselves can become a security vulnerability, extending AI safety concerns beyond isolated models to multi-agent societies. By analyzing poisoning, social conformity, and system-level verification across multiple models and network settings, this study offers timely insights into the robustness, safety, and trustworthy coordination of emerging autonomous multi-agent AI systems.</p><p>The paper [<span>3</span>] titled “Lost in the comments: Social context as a single-pass jailbreak and defense on agentic platforms” by Jeongsu Park and colleagues investigates how comments, reactions, and engagement signals surrounding harmful content influence the safety behavior of LLMs in agentic social platforms. The study demonstrates that the ambient social context, even without explicit malicious instructions, can substantially alter attack success rates, whereas ethically framed comments can reinforce safe behavior through the same contextual channel. Through controlled experiments that varied the valence, volume, and engagement of social signals, this study shows that the nature of the surrounding commentary is more important than its quantity in determining whether model safety is weakened or strengthened. This work is particularly valuable because it extends indirect prompt-injection research to the emerging setting of multi-agent platforms and provides evidence that social context can function as both an attack mechanism and a potential defensive control. Supported by comparisons with PAIR, independent rescoring with WildGuard, human adjudication, and signal-separation ablation studies, this study offers a timely and practically meaningful contribution to the secure design and operation of future agentic AI ecosystems.</p><p>While past research has primarily focused on how well AI can translate and understand multiple languages, recent studies have increasingly shifted attention toward cross-lingual safety—whether the safety alignment learned from English transfers reliably across languages.</p><p>This study [<span>4</span>] shows that the cross-lingual safety of open-weight LLMs is highly language-dependent rather than a generic vulnerability shared across all non-English languages. Through a systematic evaluation of six open-weight LLMs with a particular focus on Korean and Chinese, this study finds that safety erosion varies substantially by language and appears to be closely associated with the extent of language-specific safety alignment coverage.</p><p>Notably, this study identifies two distinct vulnerability patterns: one includes models whose safety degradation is strongly affected by translation quality, and the other includes models that remain vulnerable even under high-quality translations. It also reveals a “no man's land” between monolingual safety detectors, where mixed-language inputs can exploit a between-language blind spot. Taken together, these findings reframe multilingual AI safety as a per-language alignment problem and provide practical guidance for improving safety in real-world multilingual deployments.</p><p>The paper [<span>5</span>] titled “SF-CAMNet: A spatial–frequency fusion network for rrobust spatial domain image steganalysis” by Batool and Bashir proposes a dual-branch network that detects data hidden in digital images by fusing spatial residuals with frequency-domain features. By combining SRM filters and attention modules in the spatial branch with a wavelet-transform branch, the model uses cross-attention so that frequency cues adaptively reweight spatial features. Evaluations on BOSSBase v1.01 and BOWS2 across five adaptive steganography algorithms show competitive accuracy at a computational cost far below that of recent transformer-based baselines, offering a practical and efficient direction for digital forensics and cybersecurity.</p><p>The other two studies address the robustness and reliability of AI models under imperfect input conditions. These studies investigate how conflicting evidence and corrupted inputs can affect model behavior and how reliable performance can be maintained beyond standard evaluation settings.</p><p>The paper [<span>6</span>] titled “Diagnosing judgment flip risk in LLM fact verification under conflicting evidence” by Roh and Bae introduces a flip-rate protocol that measures how often a correct verdict of an LLM fact verifier becomes incorrect when the retrieved evidence is perturbed in terms of ratio, quantity, or order. Across six models and two datasets, even a single refuting passage flips a nontrivial share of correct judgments, and the flip rates exceed 60% on FEVER and 90% on SciFact under full conflict. This study shows that clean evidence accuracy alone is an insufficient reliability indicator, establishing evidence composition as a distinct axis of robustness evaluation.</p><p>The paper [<span>7</span>] titled “Cross-corruption distillation for robust instruction following under text corruptions” by Bae and Bae proposes cross-corruption distillation, which reframes knowledge distillation from matching a teacher's static outputs to aligning the corruption-induced logit differences between teacher and student through a KL objective. Across English and Korean multimodal benchmarks, CCD preserves competitive clean performance while improving robustness against text corruption. This study broadens the notion of knowledge distillation from transferring what a teacher predicts to transferring how it responds to input perturbations, thereby providing a behavior-level framework for training more robust multimodal models.</p><p>The following three papers further explore the integration of trustworthy AI into autonomous and safety-critical systems. These studies address human–AI interaction, reliable system validation, and the safe incorporation of AI-driven decision making in automated driving and autonomous UAV operations.</p><p>The paper [<span>8</span>] titled “STPA-based analysis of driver misuse risk scenarios in level 3 automated driving” by Lee and others proposes an analytical framework that integrates the safety of the intended functionality (SOTIF) perspective with system-theoretic process analysis (STPA) to systematically analyze driver misuse in level 3 automated driving, where safety critically depends on the driver's response to take-over requests (TOR). Rather than treating misuse as an isolated behavioral error, this study interprets it as inadequate control within the driver–system interaction structure, deriving 41 causal scenarios with four psychological states—overtrust, distrust, distraction, and interference—as causal factors and evaluating them in terms of severity, likelihood, and controllability. Clustering analysis further identifies recurring risk archetypes, such as perception failure, warning configuration error, and delayed response, showing that many high-risk scenarios stem from the driver's failure to perceive or properly respond to TORs. By providing a structured basis for understanding driver misuse risks, this study offers practical implications for driver monitoring, HMI design, and multilayer safety assistance mechanisms for trustworthy and safe automated driving.</p><p>The paper [<span>9</span>] titled “Development of two-phase framework for driver behavior classification and representative motion prototyping for reliable simulation-based driver monitoring system” by Kurniadi and others proposes a two-phase framework that automatically generates realistic driver motion baselines from in-cabin 3D skeleton data, bridging driver behavior recognition and simulation-based validation of driver monitoring systems (DMS). In phase 1, eight driver behaviors are classified using a spatial temporal graph convolutional network (ST-GCN++), validated against representative skeleton-based baselines on a subject-independent dataset, and in phase 2, the real recorded sequence closest to each class-wise mean trajectory is selected using dynamic time warping (DTW), producing a compact and verifiable motion library rather than synthetic averages. The extracted prototypes are retargeted onto MetaHuman characters in Unreal Engine, demonstrating their direct applicability to virtual scenario generation and standardized human-subject experiments. By establishing interpretable, traceable, and verifiable behavioral baselines, this study instantiates the principles of trustworthy AI in driver monitoring and contributes to reliable simulation-based safety testing for automated driving.</p><p>The paper [<span>10</span>] titled “LLM-guided semantic-aware trajectory optimization for UAV path planning using environmental context” by Lee and others proposes a semantic-aware trajectory planning framework for urban UAV operations in which a LLM serves as a high-level semantic decision module rather than a direct trajectory generator. A semantic grid map constructed from OpenStreetMap data was transformed into a differentiable semantic potential field, and the LLM adaptively modulates class-wise semantic weights according to the environmental context and mission intent, while a downstream B-spline optimizer generates smooth and collision-free trajectories. Simulation results in urban environments show that the proposed framework effectively increases clearance from semantically sensitive regions, such as schools and hospitals, compared with conventional fixed-cost planners, while keeping trajectory variations bounded under stochastic LLM outputs. By constraining the LLM to a bounded and interpretable semantic-policy interface, this study presents a principled method to combine natural-language mission reasoning with deterministic trajectory optimization, contributing to trustworthy and safe autonomous UAV operations.</p><p>The final paper highlights the role of explainability in translating high-performance AI models into trustworthy, real-world applications. It emphasizes not only how AI decisions can be made interpretable but also how their explanations can be validated by domain experts for practical use.</p><p>AI-based visual grading has the potential to improve the speed and consistency of agricultural quality assessments; however, high predictive accuracy alone is not sufficient in practice. AI decisions must also be transparent and traceable to established grading criteria, particularly when they directly affect export value and farmers' income.</p><p>To address this challenge, a study [<span>11</span>] examines explainability from two complementary perspectives: conventional post-hoc XAI methods that explain a pretrained classifier and a compositional approach that embeds interpretability directly into the grading process through segmentation, attribute classification, and rule-based decision-making. Using 4603 expert-graded clove images, the authors systematically compare explanation quality across multiple deep learning architectures and introduce the explanation energy ratio (EER) as a quantitative measure of how strongly model explanations align with the relevant object regions.</p><p>A key finding was that similar classification accuracies could mask substantial differences in explanation quality and robustness. Models that performed nearly identically in-domain can behave differently under background shifts. Importantly, this study goes beyond algorithmic evaluation by validating explanations with certified commodity graders. The compositional approach achieved the highest grade of agreement and was unanimously preferred for regulatory auditing, providing rare real-world evidence that interpretable-by-design systems can better support practical deployment.</p><p>Collectively, the studies presented in this special issue highlight the expanding scope of trustworthy and safe AI, from model-level security, robustness, and interpretability to the safety of increasingly autonomous and interconnected AI systems. Together, these underscore the importance of addressing AI safety and trustworthiness not only at the model level but also across system integration and real-world deployment. We hope that the studies presented in this special issue will stimulate further research on safer, more robust, transparent, and trustworthy AI systems. We sincerely thank all the authors, reviewers, and editorial staff of the <i>ETRI Journal</i> for their valuable contributions to this special issue.</p><p>The authors declare that there are no conflicts of interest.</p>","PeriodicalId":11901,"journal":{"name":"ETRI Journal","volume":"48 5","pages":"763-767"},"PeriodicalIF":2.0000,"publicationDate":"2026-10-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.4218/etr2.70162","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"ETRI Journal","FirstCategoryId":"94","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.4218/etr2.70162","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2026/10/15 0:00:00","PubModel":"Epub","JCR":"Q3","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0

Abstract

Recent advances in artificial intelligence (AI), particularly in large language models (LLMs) and physical robotics, have rapidly reshaped daily life and industrial applications. As these systems become more capable and autonomous, ensuring their safety, trustworthiness, robustness, and alignment with human values has become increasingly important. In response to these challenges, this special issue on “Trustworthy and Safe AI: Interpretability, Robustness, and Security” brings together recent research that advances the safety and trustworthiness of AI across diverse models, systems, and real-world applications.

This special issue received 50 submissions, reflecting the growing research interest in trustworthy and safe AI. Following a rigorous peer-review process, 11 papers were selected for publication. Together, these studies span several interconnected dimensions of the field, ranging from emerging security threats in LLMs and multi-agent systems to robustness, autonomous system safety, and explainability for practical deployment.

The following five papers focus on emerging security and safety threats across modern AI systems. These studies examine how vulnerabilities arise from adversarial attacks, multi-agent interactions, social context, multilingual settings, and hidden information while exploring the corresponding mechanisms for detection and defense.

In [1], the authors investigate a more evasive mixed clean/dirty-label X-to-X backdoor attack and propose an ensemble defense that combines complementary detectors such as Neural Cleanse, PT-RED, CEPA, and MMBD. This work is significant because it addresses both robustness and interpretability, and the ensemble improves resilience across diverse backdoor mechanisms while also supporting trigger inversion to help explain the nature of the detected attack. By jointly considering stronger adversarial attacks, generalized defense, and interpretable backdoor analysis, this study aligns well with the themes of Trustworthy and Safe AI: Interpretability, Robustness, and Security.

In [2], the authors study how a single malicious agent can compromise the collective decision-making of otherwise well-aligned LLM agents in a collaborative financial multi-agent environment and proposes a data-lineage-based systemic defense. Its main contribution is to show that the interaction dynamics themselves can become a security vulnerability, extending AI safety concerns beyond isolated models to multi-agent societies. By analyzing poisoning, social conformity, and system-level verification across multiple models and network settings, this study offers timely insights into the robustness, safety, and trustworthy coordination of emerging autonomous multi-agent AI systems.

The paper [3] titled “Lost in the comments: Social context as a single-pass jailbreak and defense on agentic platforms” by Jeongsu Park and colleagues investigates how comments, reactions, and engagement signals surrounding harmful content influence the safety behavior of LLMs in agentic social platforms. The study demonstrates that the ambient social context, even without explicit malicious instructions, can substantially alter attack success rates, whereas ethically framed comments can reinforce safe behavior through the same contextual channel. Through controlled experiments that varied the valence, volume, and engagement of social signals, this study shows that the nature of the surrounding commentary is more important than its quantity in determining whether model safety is weakened or strengthened. This work is particularly valuable because it extends indirect prompt-injection research to the emerging setting of multi-agent platforms and provides evidence that social context can function as both an attack mechanism and a potential defensive control. Supported by comparisons with PAIR, independent rescoring with WildGuard, human adjudication, and signal-separation ablation studies, this study offers a timely and practically meaningful contribution to the secure design and operation of future agentic AI ecosystems.

While past research has primarily focused on how well AI can translate and understand multiple languages, recent studies have increasingly shifted attention toward cross-lingual safety—whether the safety alignment learned from English transfers reliably across languages.

This study [4] shows that the cross-lingual safety of open-weight LLMs is highly language-dependent rather than a generic vulnerability shared across all non-English languages. Through a systematic evaluation of six open-weight LLMs with a particular focus on Korean and Chinese, this study finds that safety erosion varies substantially by language and appears to be closely associated with the extent of language-specific safety alignment coverage.

Notably, this study identifies two distinct vulnerability patterns: one includes models whose safety degradation is strongly affected by translation quality, and the other includes models that remain vulnerable even under high-quality translations. It also reveals a “no man's land” between monolingual safety detectors, where mixed-language inputs can exploit a between-language blind spot. Taken together, these findings reframe multilingual AI safety as a per-language alignment problem and provide practical guidance for improving safety in real-world multilingual deployments.

The paper [5] titled “SF-CAMNet: A spatial–frequency fusion network for rrobust spatial domain image steganalysis” by Batool and Bashir proposes a dual-branch network that detects data hidden in digital images by fusing spatial residuals with frequency-domain features. By combining SRM filters and attention modules in the spatial branch with a wavelet-transform branch, the model uses cross-attention so that frequency cues adaptively reweight spatial features. Evaluations on BOSSBase v1.01 and BOWS2 across five adaptive steganography algorithms show competitive accuracy at a computational cost far below that of recent transformer-based baselines, offering a practical and efficient direction for digital forensics and cybersecurity.

The other two studies address the robustness and reliability of AI models under imperfect input conditions. These studies investigate how conflicting evidence and corrupted inputs can affect model behavior and how reliable performance can be maintained beyond standard evaluation settings.

The paper [6] titled “Diagnosing judgment flip risk in LLM fact verification under conflicting evidence” by Roh and Bae introduces a flip-rate protocol that measures how often a correct verdict of an LLM fact verifier becomes incorrect when the retrieved evidence is perturbed in terms of ratio, quantity, or order. Across six models and two datasets, even a single refuting passage flips a nontrivial share of correct judgments, and the flip rates exceed 60% on FEVER and 90% on SciFact under full conflict. This study shows that clean evidence accuracy alone is an insufficient reliability indicator, establishing evidence composition as a distinct axis of robustness evaluation.

The paper [7] titled “Cross-corruption distillation for robust instruction following under text corruptions” by Bae and Bae proposes cross-corruption distillation, which reframes knowledge distillation from matching a teacher's static outputs to aligning the corruption-induced logit differences between teacher and student through a KL objective. Across English and Korean multimodal benchmarks, CCD preserves competitive clean performance while improving robustness against text corruption. This study broadens the notion of knowledge distillation from transferring what a teacher predicts to transferring how it responds to input perturbations, thereby providing a behavior-level framework for training more robust multimodal models.

The following three papers further explore the integration of trustworthy AI into autonomous and safety-critical systems. These studies address human–AI interaction, reliable system validation, and the safe incorporation of AI-driven decision making in automated driving and autonomous UAV operations.

The paper [8] titled “STPA-based analysis of driver misuse risk scenarios in level 3 automated driving” by Lee and others proposes an analytical framework that integrates the safety of the intended functionality (SOTIF) perspective with system-theoretic process analysis (STPA) to systematically analyze driver misuse in level 3 automated driving, where safety critically depends on the driver's response to take-over requests (TOR). Rather than treating misuse as an isolated behavioral error, this study interprets it as inadequate control within the driver–system interaction structure, deriving 41 causal scenarios with four psychological states—overtrust, distrust, distraction, and interference—as causal factors and evaluating them in terms of severity, likelihood, and controllability. Clustering analysis further identifies recurring risk archetypes, such as perception failure, warning configuration error, and delayed response, showing that many high-risk scenarios stem from the driver's failure to perceive or properly respond to TORs. By providing a structured basis for understanding driver misuse risks, this study offers practical implications for driver monitoring, HMI design, and multilayer safety assistance mechanisms for trustworthy and safe automated driving.

The paper [9] titled “Development of two-phase framework for driver behavior classification and representative motion prototyping for reliable simulation-based driver monitoring system” by Kurniadi and others proposes a two-phase framework that automatically generates realistic driver motion baselines from in-cabin 3D skeleton data, bridging driver behavior recognition and simulation-based validation of driver monitoring systems (DMS). In phase 1, eight driver behaviors are classified using a spatial temporal graph convolutional network (ST-GCN++), validated against representative skeleton-based baselines on a subject-independent dataset, and in phase 2, the real recorded sequence closest to each class-wise mean trajectory is selected using dynamic time warping (DTW), producing a compact and verifiable motion library rather than synthetic averages. The extracted prototypes are retargeted onto MetaHuman characters in Unreal Engine, demonstrating their direct applicability to virtual scenario generation and standardized human-subject experiments. By establishing interpretable, traceable, and verifiable behavioral baselines, this study instantiates the principles of trustworthy AI in driver monitoring and contributes to reliable simulation-based safety testing for automated driving.

The paper [10] titled “LLM-guided semantic-aware trajectory optimization for UAV path planning using environmental context” by Lee and others proposes a semantic-aware trajectory planning framework for urban UAV operations in which a LLM serves as a high-level semantic decision module rather than a direct trajectory generator. A semantic grid map constructed from OpenStreetMap data was transformed into a differentiable semantic potential field, and the LLM adaptively modulates class-wise semantic weights according to the environmental context and mission intent, while a downstream B-spline optimizer generates smooth and collision-free trajectories. Simulation results in urban environments show that the proposed framework effectively increases clearance from semantically sensitive regions, such as schools and hospitals, compared with conventional fixed-cost planners, while keeping trajectory variations bounded under stochastic LLM outputs. By constraining the LLM to a bounded and interpretable semantic-policy interface, this study presents a principled method to combine natural-language mission reasoning with deterministic trajectory optimization, contributing to trustworthy and safe autonomous UAV operations.

The final paper highlights the role of explainability in translating high-performance AI models into trustworthy, real-world applications. It emphasizes not only how AI decisions can be made interpretable but also how their explanations can be validated by domain experts for practical use.

AI-based visual grading has the potential to improve the speed and consistency of agricultural quality assessments; however, high predictive accuracy alone is not sufficient in practice. AI decisions must also be transparent and traceable to established grading criteria, particularly when they directly affect export value and farmers' income.

To address this challenge, a study [11] examines explainability from two complementary perspectives: conventional post-hoc XAI methods that explain a pretrained classifier and a compositional approach that embeds interpretability directly into the grading process through segmentation, attribute classification, and rule-based decision-making. Using 4603 expert-graded clove images, the authors systematically compare explanation quality across multiple deep learning architectures and introduce the explanation energy ratio (EER) as a quantitative measure of how strongly model explanations align with the relevant object regions.

A key finding was that similar classification accuracies could mask substantial differences in explanation quality and robustness. Models that performed nearly identically in-domain can behave differently under background shifts. Importantly, this study goes beyond algorithmic evaluation by validating explanations with certified commodity graders. The compositional approach achieved the highest grade of agreement and was unanimously preferred for regulatory auditing, providing rare real-world evidence that interpretable-by-design systems can better support practical deployment.

Collectively, the studies presented in this special issue highlight the expanding scope of trustworthy and safe AI, from model-level security, robustness, and interpretability to the safety of increasingly autonomous and interconnected AI systems. Together, these underscore the importance of addressing AI safety and trustworthiness not only at the model level but also across system integration and real-world deployment. We hope that the studies presented in this special issue will stimulate further research on safer, more robust, transparent, and trustworthy AI systems. We sincerely thank all the authors, reviewers, and editorial staff of the ETRI Journal for their valuable contributions to this special issue.

The authors declare that there are no conflicts of interest.

值得信赖和安全的人工智能特刊:可解释性、鲁棒性和安全性
人工智能(AI)的最新进展,特别是在大型语言模型(llm)和物理机器人方面,已经迅速重塑了日常生活和工业应用。随着这些系统变得更加强大和自主,确保它们的安全性、可靠性、稳健性以及与人类价值观的一致性变得越来越重要。为了应对这些挑战,本期“可信赖和安全的人工智能:可解释性、鲁棒性和安全性”特刊汇集了最近的研究成果,这些研究成果促进了人工智能在不同模型、系统和现实世界应用中的安全性和可信赖性。本期特刊收到了50份投稿,反映了人们对可信赖和安全的人工智能日益增长的研究兴趣。经过严格的同行评议,11篇论文被选中发表。总之,这些研究跨越了该领域的几个相互关联的维度,从法学硕士和多代理系统中出现的安全威胁到鲁棒性、自主系统安全性和实际部署的可解释性。以下五篇论文重点关注现代人工智能系统中出现的安全和安全威胁。这些研究考察了漏洞是如何从对抗性攻击、多代理交互、社会环境、多语言设置和隐藏信息中产生的,同时探索了相应的检测和防御机制。在b[1]中,作者研究了一种更具规避性的干净/脏标签混合X-to-X后门攻击,并提出了一种集成防御,该防御结合了Neural cleanup、PT-RED、CEPA和MMBD等互补检测器。这项工作很重要,因为它解决了鲁棒性和可解释性,并且集成提高了跨不同后门机制的弹性,同时还支持触发反转,以帮助解释检测到的攻击的性质。通过联合考虑更强的对抗性攻击、通用防御和可解释的后门分析,本研究与“可信赖和安全的人工智能:可解释性、鲁棒性和安全性”的主题非常一致。在b[2]中,作者研究了单个恶意代理如何在协作金融多代理环境中危及其他对齐良好的LLM代理的集体决策,并提出了基于数据谱系的系统防御。它的主要贡献是表明交互动态本身可能成为安全漏洞,将人工智能安全问题从孤立模型扩展到多智能体社会。通过分析中毒、社会一致性和跨多个模型和网络设置的系统级验证,本研究及时洞察了新兴自主多智能体AI系统的鲁棒性、安全性和可信赖的协调。Jeongsu Park及其同事发表了一篇名为《迷失在评论中:社交环境作为代理平台上的一次越狱和防御》的论文,研究了围绕有害内容的评论、反应和参与信号如何影响代理社交平台上法学硕士的安全行为。研究表明,即使没有明确的恶意指令,周围的社会环境也可以大大改变攻击成功率,而道德框架评论可以通过相同的环境渠道加强安全行为。通过改变社会信号的效价、数量和参与度的对照实验,本研究表明,在决定模型安全性是被削弱还是加强时,周围评论的性质比其数量更重要。这项工作特别有价值,因为它将间接提示注入研究扩展到多智能体平台的新兴环境,并提供证据表明,社会背景既可以作为攻击机制,也可以作为潜在的防御控制。通过与PAIR的比较、与WildGuard的独立评分、人类裁决和信号分离消融研究的支持,本研究为未来人工智能生态系统的安全设计和运行提供了及时且具有实际意义的贡献。虽然过去的研究主要集中在人工智能翻译和理解多种语言的能力上,但最近的研究越来越多地将注意力转移到跨语言安全性上——从英语中学到的安全一致性是否可靠地跨语言转移。这项研究[4]表明,开放权重llm的跨语言安全性高度依赖于语言,而不是所有非英语语言共享的通用漏洞。通过对六个开放重量法学硕士的系统评估,特别关注韩语和汉语,本研究发现,安全侵蚀因语言而异,似乎与特定语言的安全校准覆盖范围密切相关。 值得注意的是,本研究确定了两种不同的脆弱性模式:一种包括安全退化受翻译质量强烈影响的模型,另一种包括即使在高质量翻译下仍然脆弱的模型。它还揭示了单语言安全探测器之间的“无人区”,在那里,混合语言输入可以利用语言之间的盲点。综上所述,这些发现将多语言人工智能安全重新定义为每个语言的对齐问题,并为提高实际多语言部署中的安全性提供了实用指导。Batool和Bashir在题为“SF-CAMNet:用于无鲁棒空间域图像隐写分析的空间-频率融合网络”的论文中提出了一种双分支网络,该网络通过融合空间残差和频域特征来检测隐藏在数字图像中的数据。该模型将空间分支中的SRM滤波器和注意模块与小波变换分支相结合,利用交叉注意,使频率线索自适应地重新加权空间特征。对BOSSBase v1.01和BOWS2五种自适应隐写算法的评估显示,计算成本远低于最近基于变压器的基线,具有竞争力的准确性,为数字取证和网络安全提供了实用和有效的方向。另外两项研究解决了人工智能模型在不完全输入条件下的鲁棒性和可靠性问题。这些研究调查了相互冲突的证据和损坏的输入如何影响模型行为,以及如何在标准评估设置之外保持可靠的性能。卢教授和裴教授发表的论文《在相互矛盾的证据下诊断法学硕士事实验证中的判断翻转风险》(b[6])介绍了当检索到的证据在比例、数量、顺序等方面受到干扰时,衡量法学硕士事实验证者的正确判断发生错误的频率的翻转率协议。在6个模型和2个数据集中,即使是一个反驳段落,也会有相当一部分正确判断被翻转,在完全冲突的情况下,FEVER的翻转率超过60%,SciFact的翻转率超过90%。本研究表明,单纯的证据准确性是一个不充分的可靠性指标,将证据构成作为稳健性评价的一个独特轴。Bae和Bae在题为“文本腐败下稳健教学的交叉腐败蒸馏”的论文中提出了交叉腐败蒸馏,该方法将知识蒸馏从匹配教师的静态输出重新定义为通过KL目标调整教师和学生之间由腐败引起的逻辑差异。在英语和韩语的多模式基准测试中,CCD保持了具有竞争力的干净性能,同时提高了对文本损坏的鲁棒性。这项研究拓宽了知识升华的概念,从转移教师的预测到转移教师对输入扰动的反应,从而为训练更健壮的多模态模型提供了一个行为层面的框架。以下三篇论文进一步探讨了将可信赖的人工智能集成到自主和安全关键系统中的问题。这些研究涉及人类与人工智能的交互、可靠的系统验证,以及在自动驾驶和自主无人机操作中安全整合人工智能驱动的决策。Lee等人发表的题为“基于STPA的3级自动驾驶驾驶员误用风险情景分析”的论文[8]提出了一个分析框架,该框架将预期功能的安全性(SOTIF)观点与系统理论过程分析(STPA)相结合,系统地分析3级自动驾驶中的驾驶员误用,其中安全性主要取决于驾驶员对接管请求(TOR)的响应。本研究没有将滥用视为孤立的行为错误,而是将其解释为驾驶员-系统交互结构中的控制不足,推导出41种因果情景,其中包括四种心理状态——过度信任、不信任、分心和干扰——作为因果因素,并从严重性、可能性和可控性方面对其进行评估。聚类分析进一步确定了反复出现的风险原型,如感知失败、警告配置错误和延迟响应,表明许多高风险场景源于驾驶员未能感知或正确响应tor。通过为理解驾驶员误用风险提供结构化的基础,本研究为驾驶员监控、人机界面设计和多层安全辅助机制提供了实际意义,以实现可靠和安全的自动驾驶。 Kurniadi等人发表的题为“基于可靠仿真的驾驶员监控系统的驾驶员行为分类和代表性运动原型的两阶段框架开发”的论文[9]提出了一个两阶段框架,该框架可以从车内3D骨架数据自动生成真实的驾驶员运动基线,将驾驶员行为识别和基于仿真的驾驶员监控系统(DMS)验证连接起来。在第一阶段,使用时空图卷积网络(st - gcn++)对八种驾驶员行为进行分类,并在主题独立数据集上针对代表性骨架基线进行验证;在第二阶段,使用动态时间扭曲(DTW)选择最接近每个类别平均轨迹的真实记录序列,生成紧凑且可验证的运动库,而不是合成平均值。提取的原型在虚幻引擎中被重新定位到MetaHuman角色上,展示了它们对虚拟场景生成和标准化人体实验的直接适用性。通过建立可解释、可追溯和可验证的行为基线,本研究实例化了驾驶员监控中可信赖人工智能的原则,并有助于为自动驾驶提供可靠的基于模拟的安全测试。Lee等人发表的题为“基于环境上下文的无人机路径规划中LLM引导的语义感知轨迹优化”的论文[10]提出了一种用于城市无人机操作的语义感知轨迹规划框架,其中LLM作为高级语义决策模块,而不是直接的轨迹生成器。将OpenStreetMap数据构建的语义网格图转换为可微语义势场,LLM根据环境上下文和任务意图自适应调整类语义权重,而下游b样条优化器生成平滑和无碰撞的轨迹。在城市环境中的模拟结果表明,与传统的固定成本规划器相比,所提出的框架有效地增加了与语义敏感区域(如学校和医院)的间隙,同时在随机LLM输出下保持轨迹变化有限。通过将LLM约束为有界且可解释的语义策略接口,本研究提出了一种将自然语言任务推理与确定性轨迹优化相结合的原则性方法,有助于可信和安全的自主无人机操作。最后一篇论文强调了可解释性在将高性能人工智能模型转化为可信赖的现实世界应用中的作用。它不仅强调如何使人工智能决策具有可解释性,而且还强调如何通过领域专家验证其解释以用于实际应用。基于人工智能的视觉分级有可能提高农业质量评估的速度和一致性;然而,在实际应用中,仅靠较高的预测精度是不够的。人工智能决策还必须是透明的,并可追溯到既定的评级标准,特别是当它们直接影响出口价值和农民收入时。为了应对这一挑战,一项研究[11]从两个互补的角度考察了可解释性:解释预训练分类器的传统post-hoc XAI方法,以及通过分割、属性分类和基于规则的决策将可解释性直接嵌入分级过程的组合方法。使用4603张专家分级的丁香图像,作者系统地比较了多个深度学习架构的解释质量,并引入了解释能量比(EER)作为模型解释与相关对象区域对齐程度的定量度量。一个关键的发现是,相似的分类准确性可以掩盖解释质量和稳健性的实质性差异。在域内表现几乎相同的模型在背景变化下可能表现不同。重要的是,这项研究超越了算法评估,通过验证认证的商品评级者的解释。组合方法获得了最高的一致性,并被一致推荐用于监管审计,提供了罕见的真实世界证据,证明可设计解释系统可以更好地支持实际部署。总的来说,本期特刊中提出的研究强调了可信赖和安全的人工智能的范围不断扩大,从模型级安全性、鲁棒性和可解释性到日益自治和互联的人工智能系统的安全性。总之,这些都强调了解决人工智能安全性和可信度的重要性,不仅在模型层面,而且在系统集成和实际部署方面。我们希望本期特刊中提出的研究将刺激对更安全、更稳健、透明和值得信赖的人工智能系统的进一步研究。 我们衷心感谢《ETRI期刊》的所有作者、审稿人和编辑人员为本期特刊做出的宝贵贡献。作者声明不存在利益冲突。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
ETRI Journal
ETRI Journal 工程技术-电信学
CiteScore
4.00
自引率
7.10%
发文量
98
审稿时长
6.9 months
期刊介绍: ETRI Journal is an international, peer-reviewed multidisciplinary journal published bimonthly in English. The main focus of the journal is to provide an open forum to exchange innovative ideas and technology in the fields of information, telecommunications, and electronics. Key topics of interest include high-performance computing, big data analytics, cloud computing, multimedia technology, communication networks and services, wireless communications and mobile computing, material and component technology, as well as security. With an international editorial committee and experts from around the world as reviewers, ETRI Journal publishes high-quality research papers on the latest and best developments from the global community.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书