An incremental unsupervised feature extraction method based on Infomax

In recent years with the advent of big data, unsupervised feature extraction has developed rapidly, among which independent component analysis (ICA), as a classical unsupervised technique, has been widely applied in a variety of data scenarios. This paper proposes an incremental unsupervised feature extraction method based on one specific kind of ICA, i.e. Infomax. Specifically, an incremental singular value decomposition (SVD) was used in combination with the a hierarchical Infomax principle, so as to realize the rapid batch processing of data and reduce the computational complexity. Then, this method was tested with MNIST, a handwritten data set for experimental verification. The results showed that the proposed method can greatly improve the speed of feature extraction under the condition of large data volume, and ensure that the calculation results are consistent with the previous training method. Furthermore, by application in Google Speech Recognition Challenge, we verified that this method can significantly improve the training efficiency for real-world pattern recognition scenarios. The proposed method can be applied in feature extraction, data visualization and supervised learning of high-dimensional data.