Team of Tiny ANNs: A Way Towards Cost-Efficient Scalable Deep Learning

Abstract

Deep neural networks (DNNs) have latterly accomplished enormous success in various image recognition tasks. Although, training large DNN models are computationally expensive and memory intensive. So the natural idea is to do network compression and acceleration without significantly diminishing the performance of the model. In this paper, we propose a rapid and accurate method of training a neural network that has a small computation time and fewer parameters. The features are extracted using the Discrete Wavelet Transform (DWT) method. A voting-based classifier comprising a team of tiny artificial neural networks is proposed. The proposed classifier combines all the classification votes from the different sub-bands (models) to obtain the final class label, thus, achieving a similar classification accuracy of standard neural network architecture. The experiments were illustrated on benchmark data-sets of MNIST and EMNIST. On MNIST dataset, the trained models achieve the highest accuracy of 93.16 % for original and 90.44 % for Low-Low (LL) sub-band images. On the EMNIST dataset, accuracy of 90.13% for original and 87.40% for LL sub-band images has been obtained, respectively.