Advances in neural information processing systems最新文献_第6页

A Path to Simpler Models Starts With Noise. 简化模型之路从噪音开始

Advances in neural information processing systems Pub Date : 2023-12-01

Lesia Semenova, Harry Chen, Ronald Parr, Cynthia Rudin

{"title":"A Path to Simpler Models Starts With Noise.","authors":"Lesia Semenova, Harry Chen, Ronald Parr, Cynthia Rudin","doi":"","DOIUrl":"","url":null,"abstract":"The Rashomon set is the set of models that perform approximately equally well on a given dataset, and the Rashomon ratio is the fraction of all models in a given hypothesis space that are in the Rashomon set. Rashomon ratios are often large for tabular datasets in criminal justice, healthcare, lending, education, and in other areas, which has practical implications about whether simpler models can attain the same level of accuracy as more complex models. An open question is why Rashomon ratios often tend to be large. In this work, we propose and study a mechanism of the data generation process, coupled with choices usually made by the analyst during the learning process, that determines the size of the Rashomon ratio. Specifically, we demonstrate that noisier datasets lead to larger Rashomon ratios through the way that practitioners train models. Additionally, we introduce a measure called pattern diversity, which captures the average difference in predictions between distinct classification patterns in the Rashomon set, and motivate why it tends to increase with label noise. Our results explain a key aspect of why simpler models often tend to perform as well as black box models on complex, noisier datasets.","PeriodicalId":72099,"journal":{"name":"Advances in neural information processing systems","volume":"36 ","pages":"3362-3401"},"PeriodicalIF":0.0,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10993912/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140860760","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Semi-Implicit Denoising Diffusion Models (SIDDMs). 半隐式去噪扩散模型（SIDDM）。

Advances in neural information processing systems Pub Date : 2023-12-01 Epub Date: 2024-05-30

Yanwu Xu, Mingming Gong, Shaoan Xie, Wei Wei, Matthias Grundmann, Kayhan Batmanghelich, Tingbo Hou

{"title":"Semi-Implicit Denoising Diffusion Models (SIDDMs).","authors":"Yanwu Xu, Mingming Gong, Shaoan Xie, Wei Wei, Matthias Grundmann, Kayhan Batmanghelich, Tingbo Hou","doi":"","DOIUrl":"","url":null,"abstract":"Despite the proliferation of generative models, achieving fast sampling during inference without compromising sample diversity and quality remains challenging. Existing models such as Denoising Diffusion Probabilistic Models (DDPM) deliver high-quality, diverse samples but are slowed by an inherently high number of iterative steps. The Denoising Diffusion Generative Adversarial Networks (DDGAN) attempted to circumvent this limitation by integrating a GAN model for larger jumps in the diffusion process. However, DDGAN encountered scalability limitations when applied to large datasets. To address these limitations, we introduce a novel approach that tackles the problem by matching implicit and explicit factors. More specifically, our approach involves utilizing an implicit model to match the marginal distributions of noisy data and the explicit conditional distribution of the forward diffusion. This combination allows us to effectively match the joint denoising distributions. Unlike DDPM but similar to DDGAN, we do not enforce a parametric distribution for the reverse step, enabling us to take large steps during inference. Similar to the DDPM but unlike DDGAN, we take advantage of the exact form of the diffusion process. We demonstrate that our proposed method obtains comparable generative performance to diffusion-based models and vastly superior results to models with a small number of sampling steps. The code is available at https://github.com/xuyanwu/SIDDMs.","PeriodicalId":72099,"journal":{"name":"Advances in neural information processing systems","volume":"36 ","pages":"17383-17394"},"PeriodicalIF":0.0,"publicationDate":"2023-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11315220/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141918218","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Dynamic Grained Encoder for Vision Transformers 动态粒度编码器的视觉变压器

Advances in neural information processing systems Pub Date : 2023-01-10 DOI: 10.48550/arXiv.2301.03831

Lin Song, Songyang Zhang, Songtao Liu, Zeming Li, Xuming He, Hongbin Sun, Jian Sun, Nanning Zheng

引用次数: 17

Posterior Collapse and Latent Variable Non-identifiability 后塌陷和潜在变量不可识别性

Advances in neural information processing systems Pub Date : 2023-01-02 DOI: 10.48550/arXiv.2301.00537

Yixin Wang, D. Blei, J. Cunningham

{"title":"Posterior Collapse and Latent Variable Non-identifiability","authors":"Yixin Wang, D. Blei, J. Cunningham","doi":"10.48550/arXiv.2301.00537","DOIUrl":"https://doi.org/10.48550/arXiv.2301.00537","url":null,"abstract":"Variational autoencoders model high-dimensional data by positing low-dimensional latent variables that are mapped through a flexible distribution parametrized by a neural network. Unfortunately, variational autoencoders often suffer from posterior collapse: the posterior of the latent variables is equal to its prior, rendering the variational autoencoder useless as a means to produce meaningful representations. Existing approaches to posterior collapse often attribute it to the use of neural networks or optimization issues due to variational approximation. In this paper, we consider posterior collapse as a problem of latent variable non-identifiability. We prove that the posterior collapses if and only if the latent variables are non-identifiable in the generative model. This fact implies that posterior collapse is not a phenomenon specific to the use of flexible distributions or approximate inference. Rather, it can occur in classical probabilistic models even with exact inference, which we also demonstrate. Based on these results, we propose a class of latent-identifiable variational autoencoders, deep generative models which enforce identifiability without sacrificing flexibility. This model class resolves the problem of latent variable non-identifiability by leveraging bijective Brenier maps and parameterizing them with input convex neural networks, without special variational inference objectives or optimization tricks. Across synthetic and real datasets, latent-identifiable variational autoencoders outperform existing methods in mitigating posterior collapse and providing meaningful representations of the data.","PeriodicalId":72099,"journal":{"name":"Advances in neural information processing systems","volume":"1 1","pages":"5443-5455"},"PeriodicalIF":0.0,"publicationDate":"2023-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83747305","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 35

Neural Information Processing: 29th International Conference, ICONIP 2022, Virtual Event, November 22–26, 2022, Proceedings, Part III 神经信息处理:第29届国际会议，ICONIP 2022，虚拟事件，11月22日至26日，2022，会议录，第三部分

Advances in neural information processing systems Pub Date : 2023-01-01 DOI: 10.1007/978-3-031-30111-7

引用次数: 0

Neural Information Processing: 29th International Conference, ICONIP 2022, Virtual Event, November 22–26, 2022, Proceedings, Part V 神经信息处理:第29届国际会议，ICONIP 2022，虚拟事件，11月22日至26日，2022，会议录，第五部分

Advances in neural information processing systems Pub Date : 2023-01-01 DOI: 10.1007/978-981-99-1642-9

引用次数: 0

Neural Information Processing: 29th International Conference, ICONIP 2022, Virtual Event, November 22–26, 2022, Proceedings, Part II 神经信息处理:第29届国际会议，ICONIP 2022，虚拟事件，11月22日至26日，2022，会议录，第二部分

Advances in neural information processing systems Pub Date : 2023-01-01 DOI: 10.1007/978-3-031-30108-7

引用次数: 0

Neural Circuits for Fast Poisson Compressed Sensing in the Olfactory Bulb. 嗅球快速泊松压缩感知的神经回路。

Advances in neural information processing systems Pub Date : 2023-01-01

Jacob A Zavatone-Veth, Paul Masset, William L Tong, Joseph D Zak, Venkatesh N Murthy, Cengiz Pehlevan

{"title":"Neural Circuits for Fast Poisson Compressed Sensing in the Olfactory Bulb.","authors":"Jacob A Zavatone-Veth, Paul Masset, William L Tong, Joseph D Zak, Venkatesh N Murthy, Cengiz Pehlevan","doi":"","DOIUrl":"","url":null,"abstract":"Within a single sniff, the mammalian olfactory system can decode the identity and concentration of odorants wafted on turbulent plumes of air. Yet, it must do so given access only to the noisy, dimensionally-reduced representation of the odor world provided by olfactory receptor neurons. As a result, the olfactory system must solve a compressed sensing problem, relying on the fact that only a handful of the millions of possible odorants are present in a given scene. Inspired by this principle, past works have proposed normative compressed sensing models for olfactory decoding. However, these models have not captured the unique anatomy and physiology of the olfactory bulb, nor have they shown that sensing can be achieved within the 100-millisecond timescale of a single sniff. Here, we propose a rate-based Poisson compressed sensing circuit model for the olfactory bulb. This model maps onto the neuron classes of the olfactory bulb, and recapitulates salient features of their connectivity and physiology. For circuit sizes comparable to the human olfactory bulb, we show that this model can accurately detect tens of odors within the timescale of a single sniff. We also show that this model can perform Bayesian posterior sampling for accurate uncertainty estimation. Fast inference is possible only if the geometry of the neural code is chosen to match receptor properties, yielding a distributed neural code that is not axis-aligned to individual odor identities. Our results illustrate how normative modeling can help us map function onto specific neural circuits to generate new hypotheses.","PeriodicalId":72099,"journal":{"name":"Advances in neural information processing systems","volume":"36 ","pages":"64793-64828"},"PeriodicalIF":0.0,"publicationDate":"2023-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12079577/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144082422","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multi-Task Learning with Summary Statistics. 利用汇总统计进行多任务学习

Advances in neural information processing systems Pub Date : 2023-01-01 Epub Date: 2024-05-30

Parker Knight, Rui Duan

引用次数: 0

Towards robust and generalizable representations of extracellular data using contrastive learning. 利用对比学习实现细胞外数据的鲁棒性和可泛化表示。

Advances in neural information processing systems Pub Date : 2023-01-01

Ankit Vishnubhotla, Charlotte Loh, Liam Paninski, Akash Srivastava, Cole Hurwitz

引用次数: 0