Memory Technology enabling the next Artificial Intelligence revolution

Abstract

Artificial intelligence (AI), specifically Deep Learning (DL) techniques are used for real-time analytics, fraud detection, autonomous driving, and speech recognition etc. These power and data hungry DL applications on cloud and at edge has increased Deep Neural Network (DNN) complexity. Multi-tiered Compute, Memory and Storage arrangements can help push AI applications by providing faster access to high volume of data and optimizing cost. AI memory needs are quite different from traditional workloads, requiring faster access to data. DRAM manufacturers struggle with challenges like density growth, cost and bit errors. High Bandwidth Memory (HBM) and GDDR help achieve almost real time access to the memory. Each of these memories have range of system trade-offs such as density, power efficiency and bandwidth. Unlike traditional memory, Persistent memory like MRAM, Phase change memory (PCM), Resistive RAM (ReRAM), Carbon Nanotube RAM (NRAM) etc. provide non-volatility. Persistent memory has a potential to reduce the latency and cost gap between DRAM and Storage. Persistent Memory is a promising technology for driving AI but face challenges of cost, scaling and reliability. Bigger the training data set, better the inference drawn by DNN. This comes with a huge storage demand. With increase in layer count of 3D NAND and innovations in circuit design and process technology, flash enables multi-bit TLC and QLC densities. PCIe bus with SSD provides low latency and high throughput, making flash the most optimal solution for AI storage. High aspect ratio channel etch, staircase contacts, defect control etc. are some of the challenges with upcoming flash generations.