噪声和高成本密度的蒙特卡洛方法概览--应用于强化学习和 ABC

IF 1.7 3区 数学 Q1 STATISTICS & PROBABILITY
Fernando Llorente, Luca Martino, Jesse Read, David Delgado‐Gómez
{"title":"噪声和高成本密度的蒙特卡洛方法概览--应用于强化学习和 ABC","authors":"Fernando Llorente, Luca Martino, Jesse Read, David Delgado‐Gómez","doi":"10.1111/insr.12573","DOIUrl":null,"url":null,"abstract":"SummaryThis survey gives an overview of Monte Carlo methodologies using surrogate models, for dealing with densities that are intractable, costly, and/or noisy. This type of problem can be found in numerous real‐world scenarios, including stochastic optimisation and reinforcement learning, where each evaluation of a density function may incur some computationally‐expensive or even physical (real‐world activity) cost, likely to give different results each time. The surrogate model does not incur this cost, but there are important trade‐offs and considerations involved in the choice and design of such methodologies. We classify the different methodologies into three main classes and describe specific instances of algorithms under a unified notation. A modular scheme that encompasses the considered methods is also presented. A range of application scenarios is discussed, with special attention to the likelihood‐free setting and reinforcement learning. Several numerical comparisons are also provided.","PeriodicalId":14479,"journal":{"name":"International Statistical Review","volume":null,"pages":null},"PeriodicalIF":1.7000,"publicationDate":"2024-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A Survey of Monte Carlo Methods for Noisy and Costly Densities With Application to Reinforcement Learning and ABC\",\"authors\":\"Fernando Llorente, Luca Martino, Jesse Read, David Delgado‐Gómez\",\"doi\":\"10.1111/insr.12573\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"SummaryThis survey gives an overview of Monte Carlo methodologies using surrogate models, for dealing with densities that are intractable, costly, and/or noisy. This type of problem can be found in numerous real‐world scenarios, including stochastic optimisation and reinforcement learning, where each evaluation of a density function may incur some computationally‐expensive or even physical (real‐world activity) cost, likely to give different results each time. The surrogate model does not incur this cost, but there are important trade‐offs and considerations involved in the choice and design of such methodologies. We classify the different methodologies into three main classes and describe specific instances of algorithms under a unified notation. A modular scheme that encompasses the considered methods is also presented. A range of application scenarios is discussed, with special attention to the likelihood‐free setting and reinforcement learning. Several numerical comparisons are also provided.\",\"PeriodicalId\":14479,\"journal\":{\"name\":\"International Statistical Review\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":1.7000,\"publicationDate\":\"2024-05-17\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"International Statistical Review\",\"FirstCategoryId\":\"100\",\"ListUrlMain\":\"https://doi.org/10.1111/insr.12573\",\"RegionNum\":3,\"RegionCategory\":\"数学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"STATISTICS & PROBABILITY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Statistical Review","FirstCategoryId":"100","ListUrlMain":"https://doi.org/10.1111/insr.12573","RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"STATISTICS & PROBABILITY","Score":null,"Total":0}
引用次数: 0

摘要

摘要 本研究概述了使用代用模型的蒙特卡罗方法,用于处理难以处理、成本高昂和/或噪声大的密度问题。这类问题存在于现实世界的许多场景中,包括随机优化和强化学习,其中密度函数的每次评估都可能产生一些计算成本高昂甚至是物理成本(现实世界的活动)的问题,而且每次评估的结果都可能不同。代用模型不会产生这种成本,但在选择和设计此类方法时,需要进行重要的权衡和考虑。我们将不同的方法分为三大类,并用统一的符号描述算法的具体实例。此外,我们还介绍了一种包含所考虑方法的模块化方案。讨论了一系列应用场景,特别关注了无似然设置和强化学习。此外,还提供了一些数值比较。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
A Survey of Monte Carlo Methods for Noisy and Costly Densities With Application to Reinforcement Learning and ABC
SummaryThis survey gives an overview of Monte Carlo methodologies using surrogate models, for dealing with densities that are intractable, costly, and/or noisy. This type of problem can be found in numerous real‐world scenarios, including stochastic optimisation and reinforcement learning, where each evaluation of a density function may incur some computationally‐expensive or even physical (real‐world activity) cost, likely to give different results each time. The surrogate model does not incur this cost, but there are important trade‐offs and considerations involved in the choice and design of such methodologies. We classify the different methodologies into three main classes and describe specific instances of algorithms under a unified notation. A modular scheme that encompasses the considered methods is also presented. A range of application scenarios is discussed, with special attention to the likelihood‐free setting and reinforcement learning. Several numerical comparisons are also provided.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
International Statistical Review
International Statistical Review 数学-统计学与概率论
CiteScore
4.30
自引率
5.00%
发文量
52
审稿时长
>12 weeks
期刊介绍: International Statistical Review is the flagship journal of the International Statistical Institute (ISI) and of its family of Associations. It publishes papers of broad and general interest in statistics and probability. The term Review is to be interpreted broadly. The types of papers that are suitable for publication include (but are not limited to) the following: reviews/surveys of significant developments in theory, methodology, statistical computing and graphics, statistical education, and application areas; tutorials on important topics; expository papers on emerging areas of research or application; papers describing new developments and/or challenges in relevant areas; papers addressing foundational issues; papers on the history of statistics and probability; white papers on topics of importance to the profession or society; and historical assessment of seminal papers in the field and their impact.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信