TendiffPure: a convolutional tensor-train denoising diffusion model for purification

IF 2.7 3区工程技术 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS

Frontiers of Information Technology & Electronic Engineering Pub Date : 2024-01-01 DOI:10.1631/fitee.2300392

{"title":"TendiffPure: a convolutional tensor-train denoising diffusion model for purification","authors":"","doi":"10.1631/fitee.2300392","DOIUrl":null,"url":null,"abstract":"<h3>Abstract</h3> Diffusion models are effective purification methods, where the noises or adversarial attacks are removed using generative approaches before pre-existing classifiers conducting classification tasks. However, the efficiency of diffusion models is still a concern, and existing solutions are based on knowledge distillation which can jeopardize the generation quality because of the small number of generation steps. Hence, we propose TendiffPure as a tensorized and compressed diffusion model for purification. Unlike the knowledge distillation methods, we directly compress U-Nets as backbones of diffusion models using tensor-train decomposition, which reduces the number of parameters and captures more spatial information in multi-dimensional data such as images. The space complexity is reduced from O(N2) to O(NR2) with R ≤ 4 as the tensor-train rank and N as the number of channels. Experimental results show that TendiffPure can more efficiently obtain high-quality purification results and outperforms the baseline purification methods on CIFAR-10, Fashion-MNIST, and MNIST datasets for two noises and one adversarial attack.","PeriodicalId":12608,"journal":{"name":"Frontiers of Information Technology & Electronic Engineering","volume":"124 1","pages":""},"PeriodicalIF":2.7000,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Frontiers of Information Technology & Electronic Engineering","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1631/fitee.2300392","RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

Abstract

Diffusion models are effective purification methods, where the noises or adversarial attacks are removed using generative approaches before pre-existing classifiers conducting classification tasks. However, the efficiency of diffusion models is still a concern, and existing solutions are based on knowledge distillation which can jeopardize the generation quality because of the small number of generation steps. Hence, we propose TendiffPure as a tensorized and compressed diffusion model for purification. Unlike the knowledge distillation methods, we directly compress U-Nets as backbones of diffusion models using tensor-train decomposition, which reduces the number of parameters and captures more spatial information in multi-dimensional data such as images. The space complexity is reduced from O(N²) to O(NR²) with R ≤ 4 as the tensor-train rank and N as the number of channels. Experimental results show that TendiffPure can more efficiently obtain high-quality purification results and outperforms the baseline purification methods on CIFAR-10, Fashion-MNIST, and MNIST datasets for two noises and one adversarial attack.

查看原文本刊更多论文

TendiffPure：用于纯化的卷积张量训练去噪扩散模型

摘要扩散模型是一种有效的净化方法，在已有的分类器执行分类任务之前，利用生成方法去除噪声或对抗性攻击。然而，扩散模型的效率仍然是一个令人担忧的问题，现有的解决方案都是基于知识提炼，由于生成步骤较少，可能会影响生成质量。因此，我们提出了 TendiffPure，作为一种用于净化的张量压缩扩散模型。与知识蒸馏方法不同，我们使用张量-列车分解法直接压缩作为扩散模型骨干的 U-网络，从而减少了参数数量，并在图像等多维数据中捕捉到更多空间信息。空间复杂度从 O(N2) 降至 O(NR2)，其中 R ≤ 4 为张量-训练秩，N 为通道数。实验结果表明，在 CIFAR-10、Fashion-MNIST 和 MNIST 数据集上，对于两种噪声和一种对抗性攻击，TendiffPure 可以更高效地获得高质量的净化结果，其性能优于基线净化方法。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Frontiers of Information Technology & Electronic Engineering COMPUTER SCIENCE, INFORMATION SYSTEMSCOMPU-COMPUTER SCIENCE, SOFTWARE ENGINEERING

CiteScore

6.00

自引率

10.00%

发文量

1372

期刊介绍： Frontiers of Information Technology & Electronic Engineering (ISSN 2095-9184, monthly), formerly known as Journal of Zhejiang University SCIENCE C (Computers & Electronics) (2010-2014), is an international peer-reviewed journal launched by Chinese Academy of Engineering (CAE) and Zhejiang University, co-published by Springer & Zhejiang University Press. FITEE is aimed to publish the latest implementation of applications, principles, and algorithms in the broad area of Electrical and Electronic Engineering, including but not limited to Computer Science, Information Sciences, Control, Automation, Telecommunications. There are different types of articles for your choice, including research articles, review articles, science letters, perspective, new technical notes and methods, etc.