贝叶斯线性回归模型中统计推断的统计物理视角

IF 2.4 3区 物理与天体物理 Q1 Mathematics
Kazuaki Murayama
{"title":"贝叶斯线性回归模型中统计推断的统计物理视角","authors":"Kazuaki Murayama","doi":"10.1103/physreve.110.034118","DOIUrl":null,"url":null,"abstract":"This paper considers similarities between statistical physics and Bayes inference through the Bayesian linear regression model. Some similarities have been discussed previously, such as the analogy between the marginal likelihood in Bayes inference and the partition function in statistical mechanics. In particular, this paper considers the proposal to associate discrete sample size with inverse temperature [C. H. LaMont and P. A. Wiggins, <span>Phys. Rev. E</span> <b>99</b>, 052140 (2019)]. The previous study suggested that incorporating this similarity motivates the derivation of analogs of thermodynamic functions such as energy and entropy. The study also anticipated that those analogous functions have potential to describe Bayes estimation from physical points of view and to provide physical insights into mechanisms of estimation. This paper incorporates a macroscopic perspective as an asymptotics similar to the thermodynamic limit into the previous suggestion. Its motivation stems from the statistical mechanical concept of deriving thermodynamic functions that characterize macroscopic properties of macroscopic systems. This incorporation not only allows analogs of macroscopic thermodynamic functions to be considered but also suggests a candidate for an analog of inverse temperature with continuity, which is partly consistent with the previous proposal to associate the discrete sample size with inverse temperature. On the basis of this suggestion, we analyze analogs of macroscopic thermodynamic functions for a Bayesian linear regression model which is the basis of various machine learning models. We further investigate, through the behavior of these functions, how Bayes estimation is described from the perspective of physics and what kind of physical insight is obtained. As a result, the estimation of regression coefficients, which is the primary task of regression, appears to be described by the physical picture of balance between decreasing energy and increasing entropy as in equilibrium states of thermodynamic systems. More specifically we observe the physical view of Bayes inference as follows: the estimation succeeds where the effect of decreasing energy is dominant at low temperature. On the other hand, the estimation fails where the effect of increasing entropy is dominant at high temperature.","PeriodicalId":20085,"journal":{"name":"Physical review. E","volume":null,"pages":null},"PeriodicalIF":2.4000,"publicationDate":"2024-09-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Statistical physical view of statistical inference in Bayesian linear regression model\",\"authors\":\"Kazuaki Murayama\",\"doi\":\"10.1103/physreve.110.034118\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"This paper considers similarities between statistical physics and Bayes inference through the Bayesian linear regression model. Some similarities have been discussed previously, such as the analogy between the marginal likelihood in Bayes inference and the partition function in statistical mechanics. In particular, this paper considers the proposal to associate discrete sample size with inverse temperature [C. H. LaMont and P. A. Wiggins, <span>Phys. Rev. E</span> <b>99</b>, 052140 (2019)]. The previous study suggested that incorporating this similarity motivates the derivation of analogs of thermodynamic functions such as energy and entropy. The study also anticipated that those analogous functions have potential to describe Bayes estimation from physical points of view and to provide physical insights into mechanisms of estimation. This paper incorporates a macroscopic perspective as an asymptotics similar to the thermodynamic limit into the previous suggestion. Its motivation stems from the statistical mechanical concept of deriving thermodynamic functions that characterize macroscopic properties of macroscopic systems. This incorporation not only allows analogs of macroscopic thermodynamic functions to be considered but also suggests a candidate for an analog of inverse temperature with continuity, which is partly consistent with the previous proposal to associate the discrete sample size with inverse temperature. On the basis of this suggestion, we analyze analogs of macroscopic thermodynamic functions for a Bayesian linear regression model which is the basis of various machine learning models. We further investigate, through the behavior of these functions, how Bayes estimation is described from the perspective of physics and what kind of physical insight is obtained. As a result, the estimation of regression coefficients, which is the primary task of regression, appears to be described by the physical picture of balance between decreasing energy and increasing entropy as in equilibrium states of thermodynamic systems. More specifically we observe the physical view of Bayes inference as follows: the estimation succeeds where the effect of decreasing energy is dominant at low temperature. On the other hand, the estimation fails where the effect of increasing entropy is dominant at high temperature.\",\"PeriodicalId\":20085,\"journal\":{\"name\":\"Physical review. E\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":2.4000,\"publicationDate\":\"2024-09-09\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Physical review. E\",\"FirstCategoryId\":\"101\",\"ListUrlMain\":\"https://doi.org/10.1103/physreve.110.034118\",\"RegionNum\":3,\"RegionCategory\":\"物理与天体物理\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"Mathematics\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Physical review. E","FirstCategoryId":"101","ListUrlMain":"https://doi.org/10.1103/physreve.110.034118","RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"Mathematics","Score":null,"Total":0}
引用次数: 0

摘要

本文通过贝叶斯线性回归模型,探讨了统计物理学与贝叶斯推理之间的相似之处。有些相似之处之前已经讨论过,如贝叶斯推理中的边际似然与统计力学中的分区函数之间的类比。本文特别考虑了将离散样本大小与逆温度相关联的提议[C. H. LaMont and P. A. Wiggins, Phys. Rev. E 99, 052140 (2019)]。之前的研究表明,结合这种相似性可以推导出热力学函数的类似物,如能量和熵。该研究还预计,这些类似函数有可能从物理角度描述贝叶斯估计,并为估计机制提供物理见解。本文将宏观视角作为与热力学极限类似的渐近线纳入了前述建议。其动机源于统计力学概念,即导出表征宏观系统宏观特性的热力学函数。这种纳入不仅允许考虑宏观热力学函数的类似物,而且还提出了具有连续性的逆温度类似物的候选方案,这与之前提出的将离散样本大小与逆温度联系起来的建议在一定程度上是一致的。根据这一建议,我们分析了贝叶斯线性回归模型的宏观热力学函数类似物,该模型是各种机器学习模型的基础。通过这些函数的行为,我们进一步研究了如何从物理学的角度描述贝叶斯估计,以及获得了什么样的物理启示。结果发现,回归的主要任务--回归系数的估计,似乎可以用热力学系统平衡状态下能量递减和熵增加之间平衡的物理图景来描述。更具体地说,我们观察到贝叶斯推理的物理观点如下:在低温时,当能量递减效应占主导地位时,估计就会成功。另一方面,当高温时熵增效应占主导地位时,估计就会失败。
本文章由计算机程序翻译,如有差异,请以英文原文为准。

Statistical physical view of statistical inference in Bayesian linear regression model

Statistical physical view of statistical inference in Bayesian linear regression model
This paper considers similarities between statistical physics and Bayes inference through the Bayesian linear regression model. Some similarities have been discussed previously, such as the analogy between the marginal likelihood in Bayes inference and the partition function in statistical mechanics. In particular, this paper considers the proposal to associate discrete sample size with inverse temperature [C. H. LaMont and P. A. Wiggins, Phys. Rev. E 99, 052140 (2019)]. The previous study suggested that incorporating this similarity motivates the derivation of analogs of thermodynamic functions such as energy and entropy. The study also anticipated that those analogous functions have potential to describe Bayes estimation from physical points of view and to provide physical insights into mechanisms of estimation. This paper incorporates a macroscopic perspective as an asymptotics similar to the thermodynamic limit into the previous suggestion. Its motivation stems from the statistical mechanical concept of deriving thermodynamic functions that characterize macroscopic properties of macroscopic systems. This incorporation not only allows analogs of macroscopic thermodynamic functions to be considered but also suggests a candidate for an analog of inverse temperature with continuity, which is partly consistent with the previous proposal to associate the discrete sample size with inverse temperature. On the basis of this suggestion, we analyze analogs of macroscopic thermodynamic functions for a Bayesian linear regression model which is the basis of various machine learning models. We further investigate, through the behavior of these functions, how Bayes estimation is described from the perspective of physics and what kind of physical insight is obtained. As a result, the estimation of regression coefficients, which is the primary task of regression, appears to be described by the physical picture of balance between decreasing energy and increasing entropy as in equilibrium states of thermodynamic systems. More specifically we observe the physical view of Bayes inference as follows: the estimation succeeds where the effect of decreasing energy is dominant at low temperature. On the other hand, the estimation fails where the effect of increasing entropy is dominant at high temperature.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Physical review. E
Physical review. E 物理-物理:流体与等离子体
CiteScore
4.60
自引率
16.70%
发文量
0
审稿时长
3.3 months
期刊介绍: Physical Review E (PRE), broad and interdisciplinary in scope, focuses on collective phenomena of many-body systems, with statistical physics and nonlinear dynamics as the central themes of the journal. Physical Review E publishes recent developments in biological and soft matter physics including granular materials, colloids, complex fluids, liquid crystals, and polymers. The journal covers fluid dynamics and plasma physics and includes sections on computational and interdisciplinary physics, for example, complex networks.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信