Randomized dimensionality reduction of deep network features for image object recognition

Abstract

This study investigates data dimensionality reduction for image object recognition. The dimensionality reduction was applied to features extracted from an existing pre-trained Deep Neural Network (DNN) structure, the AlexNet. An analysis of the neurons in different layers of the AlexNet revealed an incremental increase in the pair-wise orthogonality between weight vectors of neurons in each layer, towards higher-level layers. This observation motivated the current study to evaluate the possibility of performing randomized dimensionality reduction by mimicking the observed orthogonality property of the high-level layers on activations of low-level layers of the AlexNet. Image object classification experiments have shown that the proposed random orthogonal projection method performed well in multiple tests, consistently outperforming the well-known statistics-based sparse random projection. Apart from being data independent, the proposed approach achieved performances comparable with the state-of-the-art techniques, but with lower computational requirements.