A New Method to Generate Virtual Samples for Solving Small Sample Set Problems

As confirmed by theory and experiments, a key factor in successfully solving a supervised learning task, especially in the case that the hypothesis is highly complex, is the number of samples available to the learner. On the other hand, in real world applications, it may not be able to provide enough number of training samples to the learner because of high acquisition cost or incapability of obtaining samples. In this paper, we propose a method addressing the problem of learning with small sample set by generating additional virtual samples. In absence of any useful prior knowledge about the functional form of the target model, we take a closer look at the distribution patterns of available samples in low dimensional subspaces and constitute the rules that each sample, including virtual samples, must obey. These rules along with other problem constraints are used as weak conditions to refine the virtual samples through an optimization process. The method is applied to two real-world learning problems. The experimental results support the efficiency of the method for solving the small sample set problems.