A Nonsynaptic Memory Based Neural Network for Hand-Written Digit Classification Using an Explainable Feature Extraction Method

Abstract

Deep learning methods have been developed for handwritten digit classification. However, these methods work as ‘black-boxes’ and need large training data. In this study, an explainable feature extraction method is developed for handwritten digit classification. The features of the digit image include horizontal, vertical, and orthogonal lines as well as full or semi-circles. In our proposed method, such features are extracted using 10 neurons as computational units. Specifically, the neurons store the features through network training and store them inside the neurons in a non-synaptic memory manner. Following that, the trained neurons are used for the retrieval of information from test images to assign them to digit categories. Our method shows an accuracy of 75 % accuracy using 0.016 % of the training data and achieves a high accuracy of 86 % using one epoch of whole training data of the MNIST dataset. To the best of our knowledge, this is the first model that stores information inside a few single neurons (i.e., non-synaptic memory) instead of storing the information in synapses of connected feed-forward layers. Due to enabling single neurons to compute individually, it is expected that such a class of neural networks can be combined with synaptic memory architectures that we expect to show higher performance compared to traditional neural networks.