Visual exploration of the inner representation learned by a convolutional neural network

We present in this paper a visual method to explore the properties of an image dataset and its internal representation learned by a convolutional neural network. We consider the inner characteristics extracted by the network just before the classification layers. We build a neighborhood graph from this vector space by connecting data together according to specific topological properties. We define typical examples of topological anomalies to be detected (isolated points, erroneous points, class boundaries). Then we propose a visualization of this graph highlighting this information and offering an overview of the graph (groups of data) as well as local details (fine topological properties). This visualization includes a representation of the images in order to let the user understand what can cause an error (errors during image acquisition, pre-processing or labeling, or errors due to the choice of the network or the learning parameters, etc.). We perform several tests with the VGG16 network on samples of standard datasets.