Exploiting Hybrid Precision for Training and Inference: A 2T-1FeFET Based Analog Synaptic Weight Cell

Xiaoyu Sun, Panni Wang, K. Ni, S. Datta, Shimeng Yu
{"title":"Exploiting Hybrid Precision for Training and Inference: A 2T-1FeFET Based Analog Synaptic Weight Cell","authors":"Xiaoyu Sun, Panni Wang, K. Ni, S. Datta, Shimeng Yu","doi":"10.1109/IEDM.2018.8614611","DOIUrl":null,"url":null,"abstract":"In-memory computing with analog non-volatile memories (NVMs) can accelerate both the in-situ training and inference of deep neural networks (DNNs) by parallelizing multiply-accumulate (MAC) operations in the analog domain. However, the in-situ training accuracy suffers from unacceptable degradation due to undesired weight-update asymmetry/nonlinearity and limited bit precision. In this work, we overcome this challenge by introducing a compact Ferroelectric FET (FeFET) based synaptic cell that exploits hybrid precision for in-situ training and inference. We propose a novel hybrid approach where we use modulated “volatile” gate voltage of FeFET to represent the least significant bits (LSBs) for symmetric/linear update during training only, and use “non-volatile” polarization states of FeFET to hold the information of most significant bits (MSBs) for inference. This design is demonstrated by the experimentally validated FeFET SPICE model and cosimulation with the TensorFlow framework. The results show that with the proposed 6-bit and 7-bit synapse design, the insitu training accuracy can achieve ∼97.3% on MNIST dataset and ∼87% on CIFAR-10 dataset, respectively, approaching the ideal software based training.","PeriodicalId":152963,"journal":{"name":"2018 IEEE International Electron Devices Meeting (IEDM)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2018-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"65","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2018 IEEE International Electron Devices Meeting (IEDM)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/IEDM.2018.8614611","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 65

Abstract

In-memory computing with analog non-volatile memories (NVMs) can accelerate both the in-situ training and inference of deep neural networks (DNNs) by parallelizing multiply-accumulate (MAC) operations in the analog domain. However, the in-situ training accuracy suffers from unacceptable degradation due to undesired weight-update asymmetry/nonlinearity and limited bit precision. In this work, we overcome this challenge by introducing a compact Ferroelectric FET (FeFET) based synaptic cell that exploits hybrid precision for in-situ training and inference. We propose a novel hybrid approach where we use modulated “volatile” gate voltage of FeFET to represent the least significant bits (LSBs) for symmetric/linear update during training only, and use “non-volatile” polarization states of FeFET to hold the information of most significant bits (MSBs) for inference. This design is demonstrated by the experimentally validated FeFET SPICE model and cosimulation with the TensorFlow framework. The results show that with the proposed 6-bit and 7-bit synapse design, the insitu training accuracy can achieve ∼97.3% on MNIST dataset and ∼87% on CIFAR-10 dataset, respectively, approaching the ideal software based training.
利用训练和推理的混合精度:基于2t - 1ffet的模拟突触权重单元
基于模拟非易失性存储器(nvm)的内存计算可以通过并行化模拟域的多重累积(MAC)运算来加速深度神经网络(dnn)的原位训练和推理。然而,由于权重更新不对称/非线性和有限的位精度,原位训练精度受到不可接受的降低。在这项工作中,我们通过引入一种紧凑的基于铁电场效应管(FeFET)的突触细胞来克服这一挑战,该细胞利用混合精度进行原位训练和推理。我们提出了一种新的混合方法,我们使用调制的FeFET的“易失性”门电压来表示训练期间对称/线性更新的最低有效位(LSBs),并使用FeFET的“非易失性”极化状态来保存最有效位(MSBs)的信息进行推理。该设计通过实验验证的FeFET SPICE模型和与TensorFlow框架的联合仿真来证明。结果表明,采用所提出的6位和7位突触设计,在MNIST数据集和CIFAR-10数据集上的原位训练准确率分别可以达到~ 97.3%和~ 87%,接近理想的基于软件的训练。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信