Combining Uncertainty Sampling Methods for Active Meta-Learning

Abstract

Meta-Learning has been applied to acquire useful knowledge to predict learning performance. Each training example in Meta-Learning (i.e. each meta-example) is related to a learning problem and stores features of the problem plus the performance obtained by a set of candidate algorithms when evaluated on the problem. Based on a set of such meta-examples, a meta-learner will be used to predict algorithm performance for new problems. The generation of a set of meta-examples can be expensive, since for each problem it is necessary to perform an empirical evaluation of the candidate algorithms. In a previous work, we proposed the Active Meta-Learning, in which Active Learning was used to reduce the set of meta-examples by selecting only the most relevant problems for meta-example generation. In the current work, we proposed the combination of different Uncertainty Sampling methods for Active Meta-Learning, considering that each individual method will provide useful information that can be combined in order to have a better assessment of problem relevance for meta-example generation. In our experiments, we observed a gain in Meta-Learning performance when the proposed method was compared to the individual active methods being combined.