Bernstein flows for flexible posteriors in variational Bayes

IF 1.4 4区 数学 Q2 STATISTICS & PROBABILITY
Oliver Dürr, Stefan Hörtling, Danil Dold, Ivonne Kovylov, Beate Sick
{"title":"Bernstein flows for flexible posteriors in variational Bayes","authors":"Oliver Dürr,&nbsp;Stefan Hörtling,&nbsp;Danil Dold,&nbsp;Ivonne Kovylov,&nbsp;Beate Sick","doi":"10.1007/s10182-024-00497-z","DOIUrl":null,"url":null,"abstract":"<div><p>Black-box variational inference (BBVI) is a technique to approximate the posterior of Bayesian models by optimization. Similar to MCMC, the user only needs to specify the model; then, the inference procedure is done automatically. In contrast to MCMC, BBVI scales to many observations, is faster for some applications, and can take advantage of highly optimized deep learning frameworks since it can be formulated as a minimization task. In the case of complex posteriors, however, other state-of-the-art BBVI approaches often yield unsatisfactory posterior approximations. This paper presents Bernstein flow variational inference (BF-VI), a robust and easy-to-use method flexible enough to approximate complex multivariate posteriors. BF-VI combines ideas from normalizing flows and Bernstein polynomial-based transformation models. In benchmark experiments, we compare BF-VI solutions with exact posteriors, MCMC solutions, and state-of-the-art BBVI methods, including normalizing flow-based BBVI. We show for low-dimensional models that BF-VI accurately approximates the true posterior; in higher-dimensional models, BF-VI compares favorably against other BBVI methods. Further, using BF-VI, we develop a Bayesian model for the semi-structured melanoma challenge data, combining a CNN model part for image data with an interpretable model part for tabular data, and demonstrate, for the first time, the use of BBVI in semi-structured models.</p></div>","PeriodicalId":55446,"journal":{"name":"Asta-Advances in Statistical Analysis","volume":null,"pages":null},"PeriodicalIF":1.4000,"publicationDate":"2024-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s10182-024-00497-z.pdf","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Asta-Advances in Statistical Analysis","FirstCategoryId":"100","ListUrlMain":"https://link.springer.com/article/10.1007/s10182-024-00497-z","RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"STATISTICS & PROBABILITY","Score":null,"Total":0}
引用次数: 0

Abstract

Black-box variational inference (BBVI) is a technique to approximate the posterior of Bayesian models by optimization. Similar to MCMC, the user only needs to specify the model; then, the inference procedure is done automatically. In contrast to MCMC, BBVI scales to many observations, is faster for some applications, and can take advantage of highly optimized deep learning frameworks since it can be formulated as a minimization task. In the case of complex posteriors, however, other state-of-the-art BBVI approaches often yield unsatisfactory posterior approximations. This paper presents Bernstein flow variational inference (BF-VI), a robust and easy-to-use method flexible enough to approximate complex multivariate posteriors. BF-VI combines ideas from normalizing flows and Bernstein polynomial-based transformation models. In benchmark experiments, we compare BF-VI solutions with exact posteriors, MCMC solutions, and state-of-the-art BBVI methods, including normalizing flow-based BBVI. We show for low-dimensional models that BF-VI accurately approximates the true posterior; in higher-dimensional models, BF-VI compares favorably against other BBVI methods. Further, using BF-VI, we develop a Bayesian model for the semi-structured melanoma challenge data, combining a CNN model part for image data with an interpretable model part for tabular data, and demonstrate, for the first time, the use of BBVI in semi-structured models.

变异贝叶斯中灵活后验的伯恩斯坦流
黑箱变分推理(BBVI)是一种通过优化近似贝叶斯模型后验的技术。与 MCMC 相似,用户只需指定模型,推理过程就会自动完成。与 MCMC 相比,BBVI 可以扩展到许多观测值,在某些应用中速度更快,而且可以利用高度优化的深度学习框架,因为它可以被表述为最小化任务。然而,在复杂后验的情况下,其他最先进的 BBVI 方法往往不能得到令人满意的后验近似值。本文介绍了伯恩斯坦流变推理(BF-VI),这是一种稳健、易用的方法,可灵活逼近复杂的多变量后验。BF-VI 结合了归一化流和基于伯恩斯坦多项式变换模型的思想。在基准实验中,我们将 BF-VI 解决方案与精确后验、MCMC 解决方案和最先进的 BBVI 方法(包括基于归一化流的 BBVI)进行了比较。结果表明,在低维模型中,BF-VI 准确地逼近了真实后验;在高维模型中,BF-VI 与其他 BBVI 方法相比更胜一筹。此外,我们利用 BF-VI 为半结构化黑色素瘤挑战数据开发了一个贝叶斯模型,将用于图像数据的 CNN 模型部分与用于表格数据的可解释模型部分相结合,并首次证明了 BBVI 在半结构化模型中的应用。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Asta-Advances in Statistical Analysis
Asta-Advances in Statistical Analysis 数学-统计学与概率论
CiteScore
2.20
自引率
14.30%
发文量
39
审稿时长
>12 weeks
期刊介绍: AStA - Advances in Statistical Analysis, a journal of the German Statistical Society, is published quarterly and presents original contributions on statistical methods and applications and review articles.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信