对数凹采样的自适应复杂性

Huanjian Zhou, Baoxiang Wang, Masashi Sugiyama
{"title":"对数凹采样的自适应复杂性","authors":"Huanjian Zhou, Baoxiang Wang, Masashi Sugiyama","doi":"arxiv-2408.13045","DOIUrl":null,"url":null,"abstract":"In large-data applications, such as the inference process of diffusion\nmodels, it is desirable to design sampling algorithms with a high degree of\nparallelization. In this work, we study the adaptive complexity of sampling,\nwhich is the minimal number of sequential rounds required to achieve sampling\ngiven polynomially many queries executed in parallel at each round. For\nunconstrained sampling, we examine distributions that are log-smooth or\nlog-Lipschitz and log strongly or non-strongly concave. We show that an almost\nlinear iteration algorithm cannot return a sample with a specific exponentially\nsmall accuracy under total variation distance. For box-constrained sampling, we\nshow that an almost linear iteration algorithm cannot return a sample with\nsup-polynomially small accuracy under total variation distance for log-concave\ndistributions. Our proof relies upon novel analysis with the characterization\nof the output for the hardness potentials based on the chain-like structure\nwith random partition and classical smoothing techniques.","PeriodicalId":501525,"journal":{"name":"arXiv - CS - Data Structures and Algorithms","volume":"12 1","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2024-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Adaptive complexity of log-concave sampling\",\"authors\":\"Huanjian Zhou, Baoxiang Wang, Masashi Sugiyama\",\"doi\":\"arxiv-2408.13045\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In large-data applications, such as the inference process of diffusion\\nmodels, it is desirable to design sampling algorithms with a high degree of\\nparallelization. In this work, we study the adaptive complexity of sampling,\\nwhich is the minimal number of sequential rounds required to achieve sampling\\ngiven polynomially many queries executed in parallel at each round. For\\nunconstrained sampling, we examine distributions that are log-smooth or\\nlog-Lipschitz and log strongly or non-strongly concave. We show that an almost\\nlinear iteration algorithm cannot return a sample with a specific exponentially\\nsmall accuracy under total variation distance. For box-constrained sampling, we\\nshow that an almost linear iteration algorithm cannot return a sample with\\nsup-polynomially small accuracy under total variation distance for log-concave\\ndistributions. Our proof relies upon novel analysis with the characterization\\nof the output for the hardness potentials based on the chain-like structure\\nwith random partition and classical smoothing techniques.\",\"PeriodicalId\":501525,\"journal\":{\"name\":\"arXiv - CS - Data Structures and Algorithms\",\"volume\":\"12 1\",\"pages\":\"\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-08-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - CS - Data Structures and Algorithms\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2408.13045\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Data Structures and Algorithms","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2408.13045","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

在大数据应用(如扩散模型的推理过程)中,设计具有高度并行性的采样算法是非常理想的。在这项工作中,我们研究了采样的自适应复杂度,即在每轮并行执行多项式查询的情况下,实现采样所需的最小连续轮数。对于无约束采样,我们研究了对数平滑或对数利普斯奇兹分布,以及对数强凹或非强凹分布。我们证明,在总变化距离下,几乎线性的迭代算法无法返回具有特定指数小精度的样本。对于盒约束采样,我们证明了对于对数凹分布,几乎线性迭代算法不能返回在总变化距离下具有向上极大值小的精度的样本。我们的证明依赖于新颖的分析,即基于具有随机分区的链状结构和经典平滑技术的硬度势的输出特征。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Adaptive complexity of log-concave sampling
In large-data applications, such as the inference process of diffusion models, it is desirable to design sampling algorithms with a high degree of parallelization. In this work, we study the adaptive complexity of sampling, which is the minimal number of sequential rounds required to achieve sampling given polynomially many queries executed in parallel at each round. For unconstrained sampling, we examine distributions that are log-smooth or log-Lipschitz and log strongly or non-strongly concave. We show that an almost linear iteration algorithm cannot return a sample with a specific exponentially small accuracy under total variation distance. For box-constrained sampling, we show that an almost linear iteration algorithm cannot return a sample with sup-polynomially small accuracy under total variation distance for log-concave distributions. Our proof relies upon novel analysis with the characterization of the output for the hardness potentials based on the chain-like structure with random partition and classical smoothing techniques.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信