Connection Reduction of DenseNet for Image Recognition

Abstract

Convolutional Neural Networks increase depth by stacking convolution layers, and deeper network models perform better in image recognition. Empirical research shows that simply stacking convolution layers does not make the network train better, and skip connection (residual learning) can improve network model performance. For the image classification tasks, models with global densely connected architectures perform well in large datasets like ImageNet, but they are not suitable for small datasets such as CIFAR-10 and SVHN. Different from dense connections, we propose two new algorithms to connect layers in this paper. Baseline is a densely connected network, and the networks connected by the two new algorithms are named ShortNet1 and ShortNet2, respectively. The experimental results of image classification on CIFAR-10 and SVHN show that ShortNet1has a 5% lower test error rate and 25% faster inference time than Baseline. ShortNet2 speeds up inference time by 40% with less loss in test accuracy. Code and pretrained models are available at https://github.com/RuiyangJu/Connection_Reduction/