An index-free sparse neural network using two-dimensional semiconductor ferroelectric field-effect transistors

IF 33.7 1区 工程技术 Q1 ENGINEERING, ELECTRICAL & ELECTRONIC
Hongkai Ning, Hengdi Wen, Yuan Meng, Zhihao Yu, Yuxiang Fu, Xilu Zou, Yilin Shen, Xiai Luo, Qiyue Zhao, Tao Zhang, Lei Liu, Shitong Zhu, Taotao Li, Weisheng Li, Li Li, Li Gao, Yi Shi, Xinran Wang
{"title":"An index-free sparse neural network using two-dimensional semiconductor ferroelectric field-effect transistors","authors":"Hongkai Ning, Hengdi Wen, Yuan Meng, Zhihao Yu, Yuxiang Fu, Xilu Zou, Yilin Shen, Xiai Luo, Qiyue Zhao, Tao Zhang, Lei Liu, Shitong Zhu, Taotao Li, Weisheng Li, Li Li, Li Gao, Yi Shi, Xinran Wang","doi":"10.1038/s41928-024-01328-4","DOIUrl":null,"url":null,"abstract":"<p>The fine-grained dynamic sparsity in biological synapses is an important element in the energy efficiency of the human brain. Emulating such sparsity in an artificial system requires off-chip memory indexing, which has a considerable energy and latency overhead. Here, we report an in-memory sparsity architecture in which index memory is moved next to individual synapses, creating a sparse neural network without external memory indexing. We use a compact building block consisting of two non-volatile ferroelectric field-effect transistors acting as a digital sparsity and an analogue weight. The network is formulated as the Hadamard product of the sparsity and weight matrices, and the hardware, which is comprised of 900 ferroelectric field-effect transistors, is based on wafer-scale chemical-vapour-deposited molybdenum disulfide integrated through back-end-of-line processes. With the system, we demonstrate key synaptic processes—including pruning, weight update and regrowth—in an unstructured and fine-grained manner. We also develop a vectorial approximate update algorithm and optimize training scheduling. Through this software–hardware co-optimization, we achieve 98.4% accuracy in an EMNIST letter recognition task under 75% sparsity. Simulations on large neural networks show a tenfold reduction in latency and a ninefold reduction in energy consumption when compared with a dense network of the same performance.</p>","PeriodicalId":19064,"journal":{"name":"Nature Electronics","volume":"19 1","pages":""},"PeriodicalIF":33.7000,"publicationDate":"2025-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Nature Electronics","FirstCategoryId":"5","ListUrlMain":"https://doi.org/10.1038/s41928-024-01328-4","RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0

Abstract

The fine-grained dynamic sparsity in biological synapses is an important element in the energy efficiency of the human brain. Emulating such sparsity in an artificial system requires off-chip memory indexing, which has a considerable energy and latency overhead. Here, we report an in-memory sparsity architecture in which index memory is moved next to individual synapses, creating a sparse neural network without external memory indexing. We use a compact building block consisting of two non-volatile ferroelectric field-effect transistors acting as a digital sparsity and an analogue weight. The network is formulated as the Hadamard product of the sparsity and weight matrices, and the hardware, which is comprised of 900 ferroelectric field-effect transistors, is based on wafer-scale chemical-vapour-deposited molybdenum disulfide integrated through back-end-of-line processes. With the system, we demonstrate key synaptic processes—including pruning, weight update and regrowth—in an unstructured and fine-grained manner. We also develop a vectorial approximate update algorithm and optimize training scheduling. Through this software–hardware co-optimization, we achieve 98.4% accuracy in an EMNIST letter recognition task under 75% sparsity. Simulations on large neural networks show a tenfold reduction in latency and a ninefold reduction in energy consumption when compared with a dense network of the same performance.

Abstract Image

利用二维半导体铁电场效应晶体管的无索引稀疏神经网络
生物突触的细粒度动态稀疏性是影响人类大脑能量效率的重要因素。在人工系统中模拟这种稀疏性需要片外存储器索引,这有相当大的能量和延迟开销。在这里,我们报告了一个内存稀疏架构,其中索引内存移动到单个突触旁边,创建了一个没有外部内存索引的稀疏神经网络。我们使用由两个非易失性铁电场效应晶体管组成的紧凑构建块作为数字稀疏性和模拟权重。该网络被表示为稀疏矩阵和权矩阵的Hadamard积,硬件由900个铁电场效应晶体管组成,基于通过后端工艺集成的晶片级化学气相沉积二硫化钼。通过该系统,我们以非结构化和细粒度的方式演示了关键的突触过程,包括修剪、权重更新和再生。我们还开发了一种向量近似更新算法并优化了训练计划。通过这种软硬件协同优化,我们在75%稀疏度下的EMNIST字母识别任务中达到了98.4%的准确率。在大型神经网络上的模拟显示,与具有相同性能的密集网络相比,延迟减少了10倍,能耗减少了9倍。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Nature Electronics
Nature Electronics Engineering-Electrical and Electronic Engineering
CiteScore
47.50
自引率
2.30%
发文量
159
期刊介绍: Nature Electronics is a comprehensive journal that publishes both fundamental and applied research in the field of electronics. It encompasses a wide range of topics, including the study of new phenomena and devices, the design and construction of electronic circuits, and the practical applications of electronics. In addition, the journal explores the commercial and industrial aspects of electronics research. The primary focus of Nature Electronics is on the development of technology and its potential impact on society. The journal incorporates the contributions of scientists, engineers, and industry professionals, offering a platform for their research findings. Moreover, Nature Electronics provides insightful commentary, thorough reviews, and analysis of the key issues that shape the field, as well as the technologies that are reshaping society. Like all journals within the prestigious Nature brand, Nature Electronics upholds the highest standards of quality. It maintains a dedicated team of professional editors and follows a fair and rigorous peer-review process. The journal also ensures impeccable copy-editing and production, enabling swift publication. Additionally, Nature Electronics prides itself on its editorial independence, ensuring unbiased and impartial reporting. In summary, Nature Electronics is a leading journal that publishes cutting-edge research in electronics. With its multidisciplinary approach and commitment to excellence, the journal serves as a valuable resource for scientists, engineers, and industry professionals seeking to stay at the forefront of advancements in the field.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信