arXiv - PHYS - Disordered Systems and Neural Networks最新文献_第7页

Quantum Hopfield Model with Dilute Memories 具有稀释记忆的量子霍普菲尔德模型

arXiv - PHYS - Disordered Systems and Neural Networks Pub Date : 2024-05-21 DOI: arxiv-2405.13240

Rongfeng Xie, Alex Kamenev

引用次数: 0

Critical feature learning in deep neural networks 深度神经网络中的关键特征学习

arXiv - PHYS - Disordered Systems and Neural Networks Pub Date : 2024-05-17 DOI: arxiv-2405.10761

Kirsten Fischer, Javed Lindner, David Dahmen, Zohar Ringel, Michael Krämer, Moritz Helias

{"title":"Critical feature learning in deep neural networks","authors":"Kirsten Fischer, Javed Lindner, David Dahmen, Zohar Ringel, Michael Krämer, Moritz Helias","doi":"arxiv-2405.10761","DOIUrl":"https://doi.org/arxiv-2405.10761","url":null,"abstract":"A key property of neural networks driving their success is their ability to\u0000learn features from data. Understanding feature learning from a theoretical\u0000viewpoint is an emerging field with many open questions. In this work we\u0000capture finite-width effects with a systematic theory of network kernels in\u0000deep non-linear neural networks. We show that the Bayesian prior of the network\u0000can be written in closed form as a superposition of Gaussian processes, whose\u0000kernels are distributed with a variance that depends inversely on the network\u0000width N . A large deviation approach, which is exact in the proportional limit\u0000for the number of data points $P = alpha N rightarrow infty$, yields a pair\u0000of forward-backward equations for the maximum a posteriori kernels in all\u0000layers at once. We study their solutions perturbatively to demonstrate how the\u0000backward propagation across layers aligns kernels with the target. An\u0000alternative field-theoretic formulation shows that kernel adaptation of the\u0000Bayesian posterior at finite-width results from fluctuations in the prior:\u0000larger fluctuations correspond to a more flexible network prior and thus enable\u0000stronger adaptation to data. We thus find a bridge between the classical\u0000edge-of-chaos NNGP theory and feature learning, exposing an intricate interplay\u0000between criticality, response functions, and feature scale.","PeriodicalId":501066,"journal":{"name":"arXiv - PHYS - Disordered Systems and Neural Networks","volume":"40 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141150703","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Generative modeling through internal high-dimensional chaotic activity 通过内部高维混沌活动生成模型

arXiv - PHYS - Disordered Systems and Neural Networks Pub Date : 2024-05-17 DOI: arxiv-2405.10822

Samantha J. Fournier, Pierfrancesco Urbani

引用次数: 0

Restoring balance: principled under/oversampling of data for optimal classification 恢复平衡：原则性数据取样不足/取样过多，实现最佳分类

arXiv - PHYS - Disordered Systems and Neural Networks Pub Date : 2024-05-15 DOI: arxiv-2405.09535

Emanuele Loffredo, Mauro Pastore, Simona Cocco, Rémi Monasson

引用次数: 0

Daydreaming Hopfield Networks and their surprising effectiveness on correlated data 白日梦 Hopfield 网络及其对相关数据的惊人功效

arXiv - PHYS - Disordered Systems and Neural Networks Pub Date : 2024-05-14 DOI: arxiv-2405.08777

Ludovica Serricchio, Dario Bocchi, Claudio Chilin, Raffaele Marino, Matteo Negri, Chiara Cammarota, Federico Ricci-Tersenghi

{"title":"Daydreaming Hopfield Networks and their surprising effectiveness on correlated data","authors":"Ludovica Serricchio, Dario Bocchi, Claudio Chilin, Raffaele Marino, Matteo Negri, Chiara Cammarota, Federico Ricci-Tersenghi","doi":"arxiv-2405.08777","DOIUrl":"https://doi.org/arxiv-2405.08777","url":null,"abstract":"To improve the storage capacity of the Hopfield model, we develop a version\u0000of the dreaming algorithm that perpetually reinforces the patterns to be stored\u0000(as in the Hebb rule), and erases the spurious memories (as in dreaming\u0000algorithms). For this reason, we called it Daydreaming. Daydreaming is not\u0000destructive and it converges asymptotically to stationary retrieval maps. When\u0000trained on random uncorrelated examples, the model shows optimal performance in\u0000terms of the size of the basins of attraction of stored examples and the\u0000quality of reconstruction. We also train the Daydreaming algorithm on\u0000correlated data obtained via the random-features model and argue that it\u0000spontaneously exploits the correlations thus increasing even further the\u0000storage capacity and the size of the basins of attraction. Moreover, the\u0000Daydreaming algorithm is also able to stabilize the features hidden in the\u0000data. Finally, we test Daydreaming on the MNIST dataset and show that it still\u0000works surprisingly well, producing attractors that are close to unseen examples\u0000and class prototypes.","PeriodicalId":501066,"journal":{"name":"arXiv - PHYS - Disordered Systems and Neural Networks","volume":"192 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140941023","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Describing the critical behavior of the Anderson transition in infinite dimension by random-matrix ensembles: logarithmic multifractality and critical localization 用随机矩阵集合描述无限维安德森转变的临界行为：对数多分形和临界定位

arXiv - PHYS - Disordered Systems and Neural Networks Pub Date : 2024-05-12 DOI: arxiv-2405.10975

Weitao Chen, Olivier Giraud, Jiangbin Gong, Gabriel Lemarié

{"title":"Describing the critical behavior of the Anderson transition in infinite dimension by random-matrix ensembles: logarithmic multifractality and critical localization","authors":"Weitao Chen, Olivier Giraud, Jiangbin Gong, Gabriel Lemarié","doi":"arxiv-2405.10975","DOIUrl":"https://doi.org/arxiv-2405.10975","url":null,"abstract":"Due to their analytical tractability, random matrix ensembles serve as robust\u0000platforms for exploring exotic phenomena in systems that are computationally\u0000demanding. Building on a companion letter [arXiv:2312.17481], this paper\u0000investigates two random matrix ensembles tailored to capture the critical\u0000behavior of the Anderson transition in infinite dimension, employing both\u0000analytical techniques and extensive numerical simulations. Our study unveils\u0000two types of critical behaviors: logarithmic multifractality and critical\u0000localization. In contrast to conventional multifractality, the novel\u0000logarithmic multifractality features eigenstate moments scaling algebraically\u0000with the logarithm of the system size. Critical localization, characterized by\u0000eigenstate moments of order $q>1/2$ converging to a finite value indicating\u0000localization, exhibits characteristic logarithmic finite-size or time effects,\u0000consistent with the critical behavior observed in random regular and\u0000Erd\"os-R'enyi graphs of effective infinite dimensionality. Using perturbative\u0000methods, we establish the existence of logarithmic multifractality and critical\u0000localization in our models. Furthermore, we explore the emergence of novel\u0000scaling behaviors in the time dynamics and spatial correlation functions. Our\u0000models provide a valuable framework for studying infinite-dimensional quantum\u0000disordered systems, and the universality of our findings enables broad\u0000applicability to systems with pronounced finite-size effects and slow dynamics,\u0000including the contentious many-body localization transition, akin to the\u0000Anderson transition in infinite dimension.","PeriodicalId":501066,"journal":{"name":"arXiv - PHYS - Disordered Systems and Neural Networks","volume":"45 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141150704","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Statistical physics of complex systems: glasses, spin glasses, continuous constraint satisfaction problems, high-dimensional inference and neural networks 复杂系统的统计物理学：玻璃、自旋玻璃、连续约束满足问题、高维推理和神经网络

arXiv - PHYS - Disordered Systems and Neural Networks Pub Date : 2024-05-10 DOI: arxiv-2405.06384

Pierfrancesco Urbani

引用次数: 0

Practical and Scalable Quantum Reservoir Computing 实用且可扩展的量子储层计算

arXiv - PHYS - Disordered Systems and Neural Networks Pub Date : 2024-05-08 DOI: arxiv-2405.04799

Chuanzhou Zhu, Peter J. Ehlers, Hendra I. Nurdin, Daniel Soh

{"title":"Practical and Scalable Quantum Reservoir Computing","authors":"Chuanzhou Zhu, Peter J. Ehlers, Hendra I. Nurdin, Daniel Soh","doi":"arxiv-2405.04799","DOIUrl":"https://doi.org/arxiv-2405.04799","url":null,"abstract":"Quantum Reservoir Computing leverages quantum systems to solve complex\u0000computational tasks with unprecedented efficiency and reduced energy\u0000consumption. This paper presents a novel QRC framework utilizing a quantum\u0000optical reservoir composed of two-level atoms within a single-mode optical\u0000cavity. Employing the Jaynes-Cummings and Tavis-Cummings models, we introduce a\u0000scalable and practically measurable reservoir that outperforms traditional\u0000classical reservoir computing in both memory retention and nonlinear data\u0000processing. We evaluate the reservoir's performance through two primary tasks:\u0000the prediction of time-series data via the Mackey-Glass task and the\u0000classification of sine-square waveforms. Our results demonstrate significant\u0000enhancements in performance with increased numbers of atoms, supported by\u0000non-destructive, continuous quantum measurements and polynomial regression\u0000techniques. This study confirms the potential of QRC to offer a scalable and\u0000efficient solution for advanced computational challenges, marking a significant\u0000step forward in the integration of quantum physics with machine learning\u0000technology.","PeriodicalId":501066,"journal":{"name":"arXiv - PHYS - Disordered Systems and Neural Networks","volume":"21 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140940958","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Exact solution of Dynamical Mean-Field Theory for a linear system with annealed disorder 退火无序线性系统的动态平均场理论精确解

arXiv - PHYS - Disordered Systems and Neural Networks Pub Date : 2024-05-08 DOI: arxiv-2405.05183

Francesco Ferraro, Christian Grilletta, Amos Maritan, Samir Suweis, Sandro Azaele

引用次数: 0

Subsystem Information Capacity in Random Circuits and Hamiltonian Dynamics 随机电路和哈密顿动力学中的子系统信息容量

arXiv - PHYS - Disordered Systems and Neural Networks Pub Date : 2024-05-08 DOI: arxiv-2405.05076

Yu-Qin Chen, Shuo Liu, Shi-Xin Zhang

{"title":"Subsystem Information Capacity in Random Circuits and Hamiltonian Dynamics","authors":"Yu-Qin Chen, Shuo Liu, Shi-Xin Zhang","doi":"arxiv-2405.05076","DOIUrl":"https://doi.org/arxiv-2405.05076","url":null,"abstract":"In this study, we explore the information capacity of open quantum systems,\u0000focusing on the effective channels formed by the subsystem of random quantum\u0000circuits and quantum Hamiltonian evolution. By analyzing the subsystem\u0000information capacity, which is closely linked to quantum coherent information\u0000of these effective quantum channels, we uncover a diverse range of dynamical\u0000and steady behaviors depending on the types of evolution. Therefore, the\u0000subsystem information capacity serves as a valuable tool for studying the\u0000intrinsic nature of various dynamical phases, such as integrable, localized,\u0000thermalized, and topological systems. We also reveal the impact of different\u0000initial information encoding schemes on information dynamics including\u0000one-to-one, one-to-many, and many-to-many. To support our findings, we provide\u0000representative examples for numerical simulations, including random quantum\u0000circuits with or without mid-circuit measurements, random Clifford Floquet\u0000circuits, free and interacting Aubry-Andr'e models, and Su-Schrieffer-Heeger\u0000models. Those numerical results are further quantitatively explained using the\u0000effective statistical model mapping and the quasiparticle picture in the cases\u0000of random circuits and non-interacting Hamiltonian dynamics, respectively.","PeriodicalId":501066,"journal":{"name":"arXiv - PHYS - Disordered Systems and Neural Networks","volume":"22 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-05-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140941031","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0