Interpretation Method of Nonlinear Multilayer Principal Component Analysis by Using Sparsity and Hierarchical Clustering

Abstract

Nonlinear multilayer principal component analysis (NMPCA) is well-known as an improved version of principal component analysis (PCA) using a five layer bottleneck neural network. NMPCA enables us to extract nonlinear hidden structure from high dimensional data, however, it has been difficult for users to understand obtained results, because trained results of NMPCA have many different locally optimal parameters depending on initial parameters. There has been no method how to find a few essential structures from many differently trained networks. This paper proposes a new interpretation method of NMPCA by extracting a few essential structures from many differently trained and locally optimal parameters. In the proposed method, firstly the weight parameters are made to be sparsely represented by LASSO training and appropriately ordered using the generalized factor loadings, then classified into a few hierarchical clusters, so that users can understand the extracted results. Its effectiveness is shown by both artificial and real world problems.