Multi-sample $$\zeta $$ -mixup: richer, more realistic synthetic samples from a p-series interpolant

IF 8.6 2区计算机科学 Q1 COMPUTER SCIENCE, THEORY & METHODS

Journal of Big Data Pub Date : 2024-03-23 DOI:10.1186/s40537-024-00898-6

{"title":"Multi-sample $$\\zeta $$ -mixup: richer, more realistic synthetic samples from a p-series interpolant","authors":"","doi":"10.1186/s40537-024-00898-6","DOIUrl":null,"url":null,"abstract":"<h3>Abstract</h3> Modern deep learning training procedures rely on model regularization techniques such as data augmentation methods, which generate training samples that increase the diversity of data and richness of label information. A popular recent method, mixup, uses convex combinations of pairs of original samples to generate new samples. However, as we show in our experiments, mixup can produce undesirable synthetic samples, where the data is sampled off the manifold and can contain incorrect labels. We propose \$\\zeta \$ -mixup, a generalization of mixup with provably and demonstrably desirable properties that allows convex combinations of \${T} \\ge 2\$ samples, leading to more realistic and diverse outputs that incorporate information from \${T}\$ original samples by using a p-series interpolant. We show that, compared to mixup, \$\\zeta \$ -mixup better preserves the intrinsic dimensionality of the original datasets, which is a desirable property for training generalizable models. Furthermore, we show that our implementation of \$\\zeta \$ -mixup is faster than mixup, and extensive evaluation on controlled synthetic and 26 diverse real-world natural and medical image classification datasets shows that \$\\zeta \$ -mixup outperforms mixup, CutMix, and traditional data augmentation techniques. The code will be released at https://github.com/kakumarabhishek/zeta-mixup.","PeriodicalId":15158,"journal":{"name":"Journal of Big Data","volume":"9 1","pages":""},"PeriodicalIF":8.6000,"publicationDate":"2024-03-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Big Data","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1186/s40537-024-00898-6","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, THEORY & METHODS","Score":null,"Total":0}

引用次数: 0

Abstract

Modern deep learning training procedures rely on model regularization techniques such as data augmentation methods, which generate training samples that increase the diversity of data and richness of label information. A popular recent method, mixup, uses convex combinations of pairs of original samples to generate new samples. However, as we show in our experiments, mixup can produce undesirable synthetic samples, where the data is sampled off the manifold and can contain incorrect labels. We propose $\zeta $ -mixup, a generalization of mixup with provably and demonstrably desirable properties that allows convex combinations of ${T} \ge 2$ samples, leading to more realistic and diverse outputs that incorporate information from ${T}$ original samples by using a p-series interpolant. We show that, compared to mixup, $\zeta $ -mixup better preserves the intrinsic dimensionality of the original datasets, which is a desirable property for training generalizable models. Furthermore, we show that our implementation of $\zeta $ -mixup is faster than mixup, and extensive evaluation on controlled synthetic and 26 diverse real-world natural and medical image classification datasets shows that $\zeta $ -mixup outperforms mixup, CutMix, and traditional data augmentation techniques. The code will be released at https://github.com/kakumarabhishek/zeta-mixup.

查看原文本刊更多论文

多样本 $$\zeta $$ -mixup：来自 p 系列插值器的更丰富、更逼真的合成样本

摘要现代深度学习训练程序依赖于数据增强方法等模型正则化技术，这些方法生成的训练样本可以增加数据的多样性和标签信息的丰富性。最近流行的一种方法是 mixup，它使用原始样本对的凸组合来生成新样本。然而，正如我们在实验中展示的那样，mixup 会产生不理想的合成样本，其中的数据采样偏离流形，可能包含不正确的标签。我们提出了 $\zeta $ -mixup，它是对 mixup 的一种概括，具有可证明和可证明的理想特性，允许 ${T} \ge 2$ 样本的凸组合，通过使用 p 系列插值法，将来自 ${T}$ 原始样本的信息纳入到更真实和多样化的输出中。我们的研究表明，与 mixup 相比，（\zeta \）-mixup 更好地保留了原始数据集的内在维度，而这正是训练可泛化模型的理想特性。此外，我们还展示了我们的 $\zeta $ -mixup 实现比 mixup 更快，在受控合成数据集和 26 个不同的真实世界自然和医学图像分类数据集上进行的广泛评估显示，$\zeta $ -mixup 优于 mixup、CutMix 和传统的数据增强技术。代码将在 https://github.com/kakumarabhishek/zeta-mixup 上发布。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Journal of Big Data Computer Science-Information Systems

CiteScore

17.80

自引率

3.70%

发文量

105

审稿时长

13 weeks

期刊介绍： The Journal of Big Data publishes high-quality, scholarly research papers, methodologies, and case studies covering a broad spectrum of topics, from big data analytics to data-intensive computing and all applications of big data research. It addresses challenges facing big data today and in the future, including data capture and storage, search, sharing, analytics, technologies, visualization, architectures, data mining, machine learning, cloud computing, distributed systems, and scalable storage. The journal serves as a seminal source of innovative material for academic researchers and practitioners alike.