Three-dimensional Memristive Deep Neural Network with Programmable Attention Mechanism

Abstract

Attention Mechanism is a critical function of a neural network-based visual system that enables the human brain can devote computing effort to process desired information, concurrently ignoring undesired background noise intentionally. Through this signal processing methodology, the brain system achieves a remarkable energy efficiency. The significance of investigating the attention mechanism is to not only potentially reveal how the human brain comprehends visual signals but also to construct an attention-oriented power-efficient neuromorphic system. This paper proposes a hypothesis for the visual attention mechanism, derives a neural network model with mathematical equations, and realizes the circuit implementation with memristor and three-dimensional (3D) monolithic integration technology.