Effects of transfer learning for handwritten digit classification in a small training sample size situation

Abstract

A deep learning approach is believed to be one of the most useful for image pattern recognition. Generally, deep learning requires a large number of samples. In particular, it is prone to overlearning when the number of training samples is small. However, it is common for practical pattern recognition problems to use a limited number of training samples. One way to design a deep neural network with such a small number of training samples is to use transfer learning. Transfer learning, known to be pre-trained with a large number of samples, and its pre-trained neural networks are expected to be applicable to other pattern recognition problems, especially when the number of training samples is small. It is difficult to develop a handwritten character classification system because handwritten characters are not always readily available and the number of samples is generally small. In this paper, we examine effects of transfer learning for handwritten digit classification under a small number of training samples. Experimental results show that transfer learning is more effective than convolutional neural networks (CNNs) in classifying handwritten digits.