Open the Black Box of Recurrent Neural Network by Decoding the Internal Dynamics

Abstract

With the development of the neural network, the complexity of the model goes far beyond the imagination. The number of neurons in the network is growing, and the black box problem requires to be solved. Although technics can record the internal dynamics of hidden neurons, the high dimension and complexity of the data bring poor interpretability. This paper introduces Tensor Component Analysis (TCA) to obtain low-dimensional information from the internal dynamics of recurrent neural networks (RNN). The proposed method extracts three interrelated neural factors: neuron factors, temporal factors, and input factors, to decode the forward propagation. This paper designs a variety of experiments to analyze the activity of RNN, and low-dimensional factors are used to explain the model's decision. The experiment shows the broad applicability of the TCA, which can accurately find the functional clustering of neurons and predict most of the classification.