Overfitting Measurement of Deep Neural Networks Using No Data

Abstract

Overfitting reduces the generalizability of deep neural networks (DNNs). Overfitting is generally detected by comparing the accuracies and losses of training and validation data; however, the detection method requires vast amounts of training data and is not always effective for forthcoming data due to the heterogeneity between training and forthcoming data. The dropout technique has been employed to prevent DNNs from overfitting, where the neurons in DNNs are invalidated randomly during their training. It has been hypothesized that this technique prevents DNNs from overfitting by restraining the co-adaptions among neurons. This hypothesis implies that overfitting of a DNN is a result of the co-adaptions among neurons and can be detected by investigating the inner representation of DNNs. Thus, we propose a method to detect overfitting of DNNs using no training and test data. The proposed method measures the degree of co-adaptions among neurons using persistent homology (PH). The proposed PH-based overfitting measure (PHOM) method constructs clique complexes on DNNs using the trained parameters of DNNs, and the one-dimensional PH investigates the co-adaptions among neurons. Thus, PHOM requires no training and test data to measure overfitting. We applied PHOM to convolutional neural networks trained for the classification problems of the CIFAR-10, SVHN, and Tiny ImageNet data sets. The experimental results demonstrate that PHOM reveals the degree of overfitting of DNNs to the training data, which suggests that PHOM enables us to filter overfitted DNNs without requiring the training and test data.