Pure-Attention-Based Multifunction Memristive Neuromorphic Circuit and System

Abstract

The use of memristive neuromorphic circuit and system is a promising solution for next-generation Artificial Intelligence (AI) computing, as it offers possibilities that go beyond conventional GPU-based artificial neural network computing platforms. However, most of the existing memristive neuromorphic circuits and systems are designed for the specific networks, which is lack of universality and flexibility. Therefore, this paper proposes a universal memristive circuit and system framework for pure-attention-based transformer networks to implement multifunction applications on edge devices. Furthermore, the verification of image recognition and speech recognition was achieved by extending the size of the memristor crossbar array macros and reconfiguring the memristor weights without changing the memristive transformer circuit and framework. This paper not only provides a universal edge implementation framework for multifunction applications of the transformer, but also offers a low-power and promising solution for the application of pure-attention-based transformers on edge devices.