与递归神经网络中的反向传播相比,神经进化产生了更集中的信息传递。

Neural Computing and Applications Pub Date : 2025-01-01 Epub Date: 2022-12-17 DOI:10.1007/s00521-022-08125-0
Arend Hintze, Christoph Adami
{"title":"与递归神经网络中的反向传播相比,神经进化产生了更集中的信息传递。","authors":"Arend Hintze, Christoph Adami","doi":"10.1007/s00521-022-08125-0","DOIUrl":null,"url":null,"abstract":"<p><p>Artificial neural networks (ANNs) are one of the most promising tools in the quest to develop general artificial intelligence. Their design was inspired by how neurons in natural brains connect and process, the only other substrate to harbor intelligence. Compared to biological brains that are sparsely connected and that form sparsely distributed representations, ANNs instead process information by connecting all nodes of one layer to all nodes of the next. In addition, modern ANNs are trained with backpropagation, while their natural counterparts have been optimized by natural evolution over eons. We study whether the training method influences how information propagates through the brain by measuring the transfer entropy, that is, the information that is transferred from one group of neurons to another. We find that while the distribution of connection weights in optimized networks is largely unaffected by the training method, neuroevolution leads to networks in which information transfer is significantly more focused on small groups of neurons (compared to those trained by backpropagation) while also being more robust to perturbations of the weights. We conclude that the specific attributes of a training method (local vs. global) can significantly affect how information is processed and relayed through the brain, even when the overall performance is similar.</p>","PeriodicalId":18925,"journal":{"name":"Neural Computing and Applications","volume":"14 1","pages":"757-767"},"PeriodicalIF":0.0000,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11757640/pdf/","citationCount":"0","resultStr":"{\"title\":\"Neuroevolution gives rise to more focused information transfer compared to backpropagation in recurrent neural networks.\",\"authors\":\"Arend Hintze, Christoph Adami\",\"doi\":\"10.1007/s00521-022-08125-0\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>Artificial neural networks (ANNs) are one of the most promising tools in the quest to develop general artificial intelligence. Their design was inspired by how neurons in natural brains connect and process, the only other substrate to harbor intelligence. Compared to biological brains that are sparsely connected and that form sparsely distributed representations, ANNs instead process information by connecting all nodes of one layer to all nodes of the next. In addition, modern ANNs are trained with backpropagation, while their natural counterparts have been optimized by natural evolution over eons. We study whether the training method influences how information propagates through the brain by measuring the transfer entropy, that is, the information that is transferred from one group of neurons to another. We find that while the distribution of connection weights in optimized networks is largely unaffected by the training method, neuroevolution leads to networks in which information transfer is significantly more focused on small groups of neurons (compared to those trained by backpropagation) while also being more robust to perturbations of the weights. We conclude that the specific attributes of a training method (local vs. global) can significantly affect how information is processed and relayed through the brain, even when the overall performance is similar.</p>\",\"PeriodicalId\":18925,\"journal\":{\"name\":\"Neural Computing and Applications\",\"volume\":\"14 1\",\"pages\":\"757-767\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2025-01-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11757640/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Neural Computing and Applications\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1007/s00521-022-08125-0\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2022/12/17 0:00:00\",\"PubModel\":\"Epub\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Neural Computing and Applications","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1007/s00521-022-08125-0","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2022/12/17 0:00:00","PubModel":"Epub","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

人工神经网络(ann)是开发通用人工智能最有前途的工具之一。他们的设计灵感来自于自然大脑中神经元的连接和处理方式,这是唯一一种孕育智慧的基质。与稀疏连接并形成稀疏分布表征的生物大脑相比,人工神经网络通过将一层的所有节点连接到下一层的所有节点来处理信息。此外,现代人工神经网络是通过反向传播训练的,而它们的自然对偶则是通过亿万年的自然进化而优化的。我们通过测量传递熵(即从一组神经元转移到另一组神经元的信息)来研究训练方法是否影响信息在大脑中的传播。我们发现,虽然优化网络中连接权值的分布在很大程度上不受训练方法的影响,但神经进化导致网络中的信息传递明显更集中于小组神经元(与反向传播训练的网络相比),同时对权值的扰动更具鲁棒性。我们得出的结论是,即使在整体表现相似的情况下,训练方法的特定属性(局部与全局)也会显著影响信息在大脑中的处理和传递方式。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Neuroevolution gives rise to more focused information transfer compared to backpropagation in recurrent neural networks.

Artificial neural networks (ANNs) are one of the most promising tools in the quest to develop general artificial intelligence. Their design was inspired by how neurons in natural brains connect and process, the only other substrate to harbor intelligence. Compared to biological brains that are sparsely connected and that form sparsely distributed representations, ANNs instead process information by connecting all nodes of one layer to all nodes of the next. In addition, modern ANNs are trained with backpropagation, while their natural counterparts have been optimized by natural evolution over eons. We study whether the training method influences how information propagates through the brain by measuring the transfer entropy, that is, the information that is transferred from one group of neurons to another. We find that while the distribution of connection weights in optimized networks is largely unaffected by the training method, neuroevolution leads to networks in which information transfer is significantly more focused on small groups of neurons (compared to those trained by backpropagation) while also being more robust to perturbations of the weights. We conclude that the specific attributes of a training method (local vs. global) can significantly affect how information is processed and relayed through the brain, even when the overall performance is similar.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信