{"title":"密集联想记忆中的顺序学习。","authors":"Hayden McAlister;Anthony Robins;Lech Szymanski","doi":"10.1162/neco.a.20","DOIUrl":null,"url":null,"abstract":"Sequential learning involves learning tasks in a sequence and proves challenging for most neural networks. Biological neural networks regularly succeed at the sequential learning challenge and are even capable of transferring knowledge both forward and backward between tasks. Artificial neural networks often totally fail to transfer performance between tasks and regularly suffer from degraded performance or catastrophic forgetting on previous tasks. Models of associative memory have been used to investigate the discrepancy between biological and artificial neural networks due to their biological ties and inspirations, of which the Hopfield network is the most studied model. The dense associative memory (DAM), or modern Hopfield network, generalizes the Hopfield network, allowing for greater capacities and prototype learning behaviors while still retaining the associative memory structure. We give a substantial review of the sequential learning space with particular respect to the Hopfield network and associative memories. We present the first published benchmarks of sequential learning in the DAM using various sequential learning techniques and analyze the results of the sequential learning to demonstrate previously unseen transitions in the behavior of the DAM. This letter also discusses the departure from biological plausibility that may affect the utility of the DAM as a tool for studying biological neural networks. We present our findings, including the effectiveness of a range of state-of-the-art sequential learning methods when applied to the DAM, and use these methods to further the understanding of DAM properties and behaviors.","PeriodicalId":54731,"journal":{"name":"Neural Computation","volume":"37 10","pages":"1877-1924"},"PeriodicalIF":2.1000,"publicationDate":"2025-09-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Sequential Learning in the Dense Associative Memory\",\"authors\":\"Hayden McAlister;Anthony Robins;Lech Szymanski\",\"doi\":\"10.1162/neco.a.20\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Sequential learning involves learning tasks in a sequence and proves challenging for most neural networks. Biological neural networks regularly succeed at the sequential learning challenge and are even capable of transferring knowledge both forward and backward between tasks. Artificial neural networks often totally fail to transfer performance between tasks and regularly suffer from degraded performance or catastrophic forgetting on previous tasks. Models of associative memory have been used to investigate the discrepancy between biological and artificial neural networks due to their biological ties and inspirations, of which the Hopfield network is the most studied model. The dense associative memory (DAM), or modern Hopfield network, generalizes the Hopfield network, allowing for greater capacities and prototype learning behaviors while still retaining the associative memory structure. We give a substantial review of the sequential learning space with particular respect to the Hopfield network and associative memories. We present the first published benchmarks of sequential learning in the DAM using various sequential learning techniques and analyze the results of the sequential learning to demonstrate previously unseen transitions in the behavior of the DAM. This letter also discusses the departure from biological plausibility that may affect the utility of the DAM as a tool for studying biological neural networks. We present our findings, including the effectiveness of a range of state-of-the-art sequential learning methods when applied to the DAM, and use these methods to further the understanding of DAM properties and behaviors.\",\"PeriodicalId\":54731,\"journal\":{\"name\":\"Neural Computation\",\"volume\":\"37 10\",\"pages\":\"1877-1924\"},\"PeriodicalIF\":2.1000,\"publicationDate\":\"2025-09-22\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Neural Computation\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/11180096/\",\"RegionNum\":4,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neural Computation","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/11180096/","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
Sequential Learning in the Dense Associative Memory
Sequential learning involves learning tasks in a sequence and proves challenging for most neural networks. Biological neural networks regularly succeed at the sequential learning challenge and are even capable of transferring knowledge both forward and backward between tasks. Artificial neural networks often totally fail to transfer performance between tasks and regularly suffer from degraded performance or catastrophic forgetting on previous tasks. Models of associative memory have been used to investigate the discrepancy between biological and artificial neural networks due to their biological ties and inspirations, of which the Hopfield network is the most studied model. The dense associative memory (DAM), or modern Hopfield network, generalizes the Hopfield network, allowing for greater capacities and prototype learning behaviors while still retaining the associative memory structure. We give a substantial review of the sequential learning space with particular respect to the Hopfield network and associative memories. We present the first published benchmarks of sequential learning in the DAM using various sequential learning techniques and analyze the results of the sequential learning to demonstrate previously unseen transitions in the behavior of the DAM. This letter also discusses the departure from biological plausibility that may affect the utility of the DAM as a tool for studying biological neural networks. We present our findings, including the effectiveness of a range of state-of-the-art sequential learning methods when applied to the DAM, and use these methods to further the understanding of DAM properties and behaviors.
期刊介绍:
Neural Computation is uniquely positioned at the crossroads between neuroscience and TMCS and welcomes the submission of original papers from all areas of TMCS, including: Advanced experimental design; Analysis of chemical sensor data; Connectomic reconstructions; Analysis of multielectrode and optical recordings; Genetic data for cell identity; Analysis of behavioral data; Multiscale models; Analysis of molecular mechanisms; Neuroinformatics; Analysis of brain imaging data; Neuromorphic engineering; Principles of neural coding, computation, circuit dynamics, and plasticity; Theories of brain function.