Robust Neural Network Training Using Inverted Probability Distribution

Abstract

This paper presents strategies to tweak the probability distribution of the data set to bias the training process of a neural network for a better learning outcome. For a real-world problem, provided that the probability distribution of the population can be assumed, the training set can be sampled from the population in such a way that its probability distribution satisfies certain targeted characteristics. For example, if the boundary between classes is critical to the training outcome, a larger proportion of training data may be drawn from the area around the boundaries. On the other hand, if the learning outcome is aimed at resembling a common concept encoded in the training set, learning from the data near the norm may be more effective. In order to explore the effectiveness of the various strategies, the concept was applied to two problems: 3-spiral and wine quality. Experimental results suggest that, whether the problem requires an emphasis on classifying boundary or recognizing the central pattern, our novel sampling strategy – inverted probability distribution – performs exceptionally well.