Nrat: towards adversarial training with inherent label noise

IF 4.3 3区计算机科学 Q2 COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Machine Learning Pub Date : 2024-01-10 DOI:10.1007/s10994-023-06437-3

{"title":"Nrat: towards adversarial training with inherent label noise","authors":"","doi":"10.1007/s10994-023-06437-3","DOIUrl":null,"url":null,"abstract":"<h3>Abstract</h3> <p>Adversarial training (AT) has been widely recognized as the most effective defense approach against adversarial attacks on deep neural networks and it is formulated as a min-max optimization. Most AT algorithms are geared towards research-oriented datasets such as MNIST, CIFAR10, etc., where the labels are generally correct. However, noisy labels, e.g., mislabelling, are inevitable in real-world datasets. In this paper, we investigate AT with inherent label noise, where the training dataset itself contains mislabeled samples. We first empirically show that the performance of AT typically degrades as the label noise rate increases. Then, we propose a <em>Noisy-Robust Adversarial Training</em> (NRAT) algorithm, which leverages the recent advancements in learning with noisy labels to enhance the performance of AT in the presence of label noise. For experimental comparison, we consider two essential metrics in AT: (i) trade-off between natural and robust accuracy; (ii) robust overfitting. Our experiments show that NRAT’s performance is on par with, or better than, the state-of-the-art AT methods on both evaluation metrics. Our code is publicly available at: https://github.com/TrustAI/NRAT.</p>","PeriodicalId":49900,"journal":{"name":"Machine Learning","volume":"101 1","pages":""},"PeriodicalIF":4.3000,"publicationDate":"2024-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Machine Learning","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s10994-023-06437-3","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}

引用次数: 0

Abstract

Adversarial training (AT) has been widely recognized as the most effective defense approach against adversarial attacks on deep neural networks and it is formulated as a min-max optimization. Most AT algorithms are geared towards research-oriented datasets such as MNIST, CIFAR10, etc., where the labels are generally correct. However, noisy labels, e.g., mislabelling, are inevitable in real-world datasets. In this paper, we investigate AT with inherent label noise, where the training dataset itself contains mislabeled samples. We first empirically show that the performance of AT typically degrades as the label noise rate increases. Then, we propose a Noisy-Robust Adversarial Training (NRAT) algorithm, which leverages the recent advancements in learning with noisy labels to enhance the performance of AT in the presence of label noise. For experimental comparison, we consider two essential metrics in AT: (i) trade-off between natural and robust accuracy; (ii) robust overfitting. Our experiments show that NRAT’s performance is on par with, or better than, the state-of-the-art AT methods on both evaluation metrics. Our code is publicly available at: https://github.com/TrustAI/NRAT.

查看原文本刊更多论文

Nrat：利用固有标签噪声进行对抗训练

摘要对抗训练（AT）已被广泛认为是对抗深度神经网络对抗攻击的最有效防御方法，它被表述为最小最大优化。大多数对抗训练算法都是针对以研究为导向的数据集，如 MNIST、CIFAR10 等，这些数据集的标签一般都是正确的。然而，在现实世界的数据集中，噪声标签（如错误标签）是不可避免的。在本文中，我们研究了具有固有标签噪声的 AT，即训练数据集本身包含错误标签样本。我们首先从经验上证明，随着标签噪声率的增加，AT 的性能通常会下降。然后，我们提出了一种噪声稳健对抗训练（NRAT）算法，该算法充分利用了最近在利用噪声标签学习方面取得的进步，从而在存在标签噪声的情况下提高对抗训练的性能。为了进行实验比较，我们考虑了对抗训练中的两个基本指标：(i) 自然准确性和鲁棒准确性之间的权衡；(ii) 鲁棒过拟合。我们的实验表明，NRAT 在这两项评价指标上的表现与最先进的 AT 方法相当，甚至更好。我们的代码可在 https://github.com/TrustAI/NRAT 公开获取。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Machine Learning 工程技术-计算机：人工智能

CiteScore

11.00

自引率

2.70%

发文量

162

审稿时长

3 months

期刊介绍： Machine Learning serves as a global platform dedicated to computational approaches in learning. The journal reports substantial findings on diverse learning methods applied to various problems, offering support through empirical studies, theoretical analysis, or connections to psychological phenomena. It demonstrates the application of learning methods to solve significant problems and aims to enhance the conduct of machine learning research with a focus on verifiable and replicable evidence in published papers.