{"title":"能量-信息权衡引发横向预测编码中的连续和不连续相变","authors":"Zhen-Ye Huang, Ruyi Zhou, Miao Huang, Hai-Jun Zhou","doi":"10.1007/s11433-024-2341-2","DOIUrl":null,"url":null,"abstract":"<p>Lateral predictive coding is a recurrent neural network that creates energy-efficient internal representations by exploiting statistical regularity in sensory inputs. Here, we analytically investigate the trade-off between information robustness and energy in a linear model of lateral predictive coding and numerically minimize a free energy quantity. We observed several phase transitions in the synaptic weight matrix, particularly a continuous transition that breaks reciprocity and permutation symmetry and builds cyclic dominance and a discontinuous transition with the associated sudden emergence of tight balance between excitatory and inhibitory interactions. The optimal network follows an ideal gas law over an extended temperature range and saturates the efficiency upper bound of energy use. These results provide theoretical insights into the emergence and evolution of complex internal models in predictive processing systems.</p>","PeriodicalId":774,"journal":{"name":"Science China Physics, Mechanics & Astronomy","volume":null,"pages":null},"PeriodicalIF":6.4000,"publicationDate":"2024-05-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Energy-information trade-off induces continuous and discontinuous phase transitions in lateral predictive coding\",\"authors\":\"Zhen-Ye Huang, Ruyi Zhou, Miao Huang, Hai-Jun Zhou\",\"doi\":\"10.1007/s11433-024-2341-2\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p>Lateral predictive coding is a recurrent neural network that creates energy-efficient internal representations by exploiting statistical regularity in sensory inputs. Here, we analytically investigate the trade-off between information robustness and energy in a linear model of lateral predictive coding and numerically minimize a free energy quantity. We observed several phase transitions in the synaptic weight matrix, particularly a continuous transition that breaks reciprocity and permutation symmetry and builds cyclic dominance and a discontinuous transition with the associated sudden emergence of tight balance between excitatory and inhibitory interactions. The optimal network follows an ideal gas law over an extended temperature range and saturates the efficiency upper bound of energy use. These results provide theoretical insights into the emergence and evolution of complex internal models in predictive processing systems.</p>\",\"PeriodicalId\":774,\"journal\":{\"name\":\"Science China Physics, Mechanics & Astronomy\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":6.4000,\"publicationDate\":\"2024-05-07\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Science China Physics, Mechanics & Astronomy\",\"FirstCategoryId\":\"101\",\"ListUrlMain\":\"https://doi.org/10.1007/s11433-024-2341-2\",\"RegionNum\":1,\"RegionCategory\":\"物理与天体物理\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"PHYSICS, MULTIDISCIPLINARY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Science China Physics, Mechanics & Astronomy","FirstCategoryId":"101","ListUrlMain":"https://doi.org/10.1007/s11433-024-2341-2","RegionNum":1,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"PHYSICS, MULTIDISCIPLINARY","Score":null,"Total":0}
Energy-information trade-off induces continuous and discontinuous phase transitions in lateral predictive coding
Lateral predictive coding is a recurrent neural network that creates energy-efficient internal representations by exploiting statistical regularity in sensory inputs. Here, we analytically investigate the trade-off between information robustness and energy in a linear model of lateral predictive coding and numerically minimize a free energy quantity. We observed several phase transitions in the synaptic weight matrix, particularly a continuous transition that breaks reciprocity and permutation symmetry and builds cyclic dominance and a discontinuous transition with the associated sudden emergence of tight balance between excitatory and inhibitory interactions. The optimal network follows an ideal gas law over an extended temperature range and saturates the efficiency upper bound of energy use. These results provide theoretical insights into the emergence and evolution of complex internal models in predictive processing systems.
期刊介绍:
Science China Physics, Mechanics & Astronomy, an academic journal cosponsored by the Chinese Academy of Sciences and the National Natural Science Foundation of China, and published by Science China Press, is committed to publishing high-quality, original results in both basic and applied research.
Science China Physics, Mechanics & Astronomy, is published in both print and electronic forms. It is indexed by Science Citation Index.
Categories of articles:
Reviews summarize representative results and achievements in a particular topic or an area, comment on the current state of research, and advise on the research directions. The author’s own opinion and related discussion is requested.
Research papers report on important original results in all areas of physics, mechanics and astronomy.
Brief reports present short reports in a timely manner of the latest important results.