Yuhao Sun, Wantong Liao, Jinhao Li, Xinche Zhang, Guan Wang, Zhiyuan Ma, Sen Song
{"title":"基于随机释放可塑性的奖励优化学习。","authors":"Yuhao Sun, Wantong Liao, Jinhao Li, Xinche Zhang, Guan Wang, Zhiyuan Ma, Sen Song","doi":"10.3389/fncir.2025.1618506","DOIUrl":null,"url":null,"abstract":"<p><p>Synaptic plasticity underlies adaptive learning in neural systems, offering a biologically plausible framework for reward-driven learning. However, a question remains: how can plasticity rules achieve robustness and effectiveness comparable to error backpropagation? In this study, we introduce Reward-Optimized Stochastic Release Plasticity (RSRP), a learning framework where synaptic release is modeled as a parameterized distribution. Utilizing natural gradient estimation, we derive a synaptic plasticity learning rule that effectively adapts to maximize reward signals. Our approach achieves competitive performance and demonstrates stability in reinforcement learning, comparable to Proximal Policy Optimization (PPO), while attaining accuracy comparable with error backpropagation in digit classification. Additionally, we identify reward regularization as a key stabilizing mechanism and validate our method in biologically plausible networks. Our findings suggest that RSRP offers a robust and effective plasticity learning rule, especially in a discontinuous reinforcement learning paradigm, with potential implications for both artificial intelligence and experimental neuroscience.</p>","PeriodicalId":12498,"journal":{"name":"Frontiers in Neural Circuits","volume":"19 ","pages":"1618506"},"PeriodicalIF":3.0000,"publicationDate":"2025-08-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12390965/pdf/","citationCount":"0","resultStr":"{\"title\":\"Reward-optimizing learning using stochastic release plasticity.\",\"authors\":\"Yuhao Sun, Wantong Liao, Jinhao Li, Xinche Zhang, Guan Wang, Zhiyuan Ma, Sen Song\",\"doi\":\"10.3389/fncir.2025.1618506\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>Synaptic plasticity underlies adaptive learning in neural systems, offering a biologically plausible framework for reward-driven learning. However, a question remains: how can plasticity rules achieve robustness and effectiveness comparable to error backpropagation? In this study, we introduce Reward-Optimized Stochastic Release Plasticity (RSRP), a learning framework where synaptic release is modeled as a parameterized distribution. Utilizing natural gradient estimation, we derive a synaptic plasticity learning rule that effectively adapts to maximize reward signals. Our approach achieves competitive performance and demonstrates stability in reinforcement learning, comparable to Proximal Policy Optimization (PPO), while attaining accuracy comparable with error backpropagation in digit classification. Additionally, we identify reward regularization as a key stabilizing mechanism and validate our method in biologically plausible networks. Our findings suggest that RSRP offers a robust and effective plasticity learning rule, especially in a discontinuous reinforcement learning paradigm, with potential implications for both artificial intelligence and experimental neuroscience.</p>\",\"PeriodicalId\":12498,\"journal\":{\"name\":\"Frontiers in Neural Circuits\",\"volume\":\"19 \",\"pages\":\"1618506\"},\"PeriodicalIF\":3.0000,\"publicationDate\":\"2025-08-14\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12390965/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Frontiers in Neural Circuits\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.3389/fncir.2025.1618506\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2025/1/1 0:00:00\",\"PubModel\":\"eCollection\",\"JCR\":\"Q2\",\"JCRName\":\"NEUROSCIENCES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Frontiers in Neural Circuits","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.3389/fncir.2025.1618506","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2025/1/1 0:00:00","PubModel":"eCollection","JCR":"Q2","JCRName":"NEUROSCIENCES","Score":null,"Total":0}
Reward-optimizing learning using stochastic release plasticity.
Synaptic plasticity underlies adaptive learning in neural systems, offering a biologically plausible framework for reward-driven learning. However, a question remains: how can plasticity rules achieve robustness and effectiveness comparable to error backpropagation? In this study, we introduce Reward-Optimized Stochastic Release Plasticity (RSRP), a learning framework where synaptic release is modeled as a parameterized distribution. Utilizing natural gradient estimation, we derive a synaptic plasticity learning rule that effectively adapts to maximize reward signals. Our approach achieves competitive performance and demonstrates stability in reinforcement learning, comparable to Proximal Policy Optimization (PPO), while attaining accuracy comparable with error backpropagation in digit classification. Additionally, we identify reward regularization as a key stabilizing mechanism and validate our method in biologically plausible networks. Our findings suggest that RSRP offers a robust and effective plasticity learning rule, especially in a discontinuous reinforcement learning paradigm, with potential implications for both artificial intelligence and experimental neuroscience.
期刊介绍:
Frontiers in Neural Circuits publishes rigorously peer-reviewed research on the emergent properties of neural circuits - the elementary modules of the brain. Specialty Chief Editors Takao K. Hensch and Edward Ruthazer at Harvard University and McGill University respectively, are supported by an outstanding Editorial Board of international experts. This multidisciplinary open-access journal is at the forefront of disseminating and communicating scientific knowledge and impactful discoveries to researchers, academics and the public worldwide.
Frontiers in Neural Circuits launched in 2011 with great success and remains a "central watering hole" for research in neural circuits, serving the community worldwide to share data, ideas and inspiration. Articles revealing the anatomy, physiology, development or function of any neural circuitry in any species (from sponges to humans) are welcome. Our common thread seeks the computational strategies used by different circuits to link their structure with function (perceptual, motor, or internal), the general rules by which they operate, and how their particular designs lead to the emergence of complex properties and behaviors. Submissions focused on synaptic, cellular and connectivity principles in neural microcircuits using multidisciplinary approaches, especially newer molecular, developmental and genetic tools, are encouraged. Studies with an evolutionary perspective to better understand how circuit design and capabilities evolved to produce progressively more complex properties and behaviors are especially welcome. The journal is further interested in research revealing how plasticity shapes the structural and functional architecture of neural circuits.