Optimizing Convolutional Neural Networks for low-resource devices

Abstract

Convolutional neural networks are effective supervised learning models which are widely used nowadays in various applications ranging from computer vision tasks such as image detection and classification, image captioning, to video classification. Even if the convolutional models are highly performant, a major drawback is given by their computationally expensiveness from the viewpoint of the required memory, additions and multiplications operations and thus are hardly portable on limited-resource devices. The purpose of this paper is to demonstrate the applicability of convolutional neural networks for low resource devices and to study their performance in real life scenarios. In this respect, with the major goal of preserving the performance, we propose a convolutional neural network model, called SimpLeNet, using distillation for image tagging that can run on low-resource devices such as smartphones, smartwatches, tablets or TVs. Experiments performed on MNIST data set for image classification emphasize the effectiveness of SimpLeNet, both in terms of model’s size reduction, as well as in terms of classification accuracy