Meta-Learning With Task-Adaptive Selection

Abstract

The gradient-based meta-learning algorithm gains meta-learning parameters from a pool of tasks. Starting from the obtained meta-learning parameters, it can achieve better results through fast fine-tuning with only a few gradient descent updates. The two-layer meta-learning approach that shares initialization parameters has achieved good results in solving few-shot learning domain. However, in the training of multiple similar tasks in the inner layer, the difficulty and benefits of the tasks have been consistently overlooked, resulting in conflicts between tasks and ultimately compromising the model to unexpected positions. Therefore, this paper proposes a task-adaptive selection meta-learning algorithm called TSML. Specifically, we construct a task selection trainer to assess the difficulty of tasks and calculate their future benefits. Designing more optimal training strategies for each task based on difficulty and benefit, altering the current compromise in multi-task settings, and balancing the impact of tasks on meta-learning parameters. Additionally, the outer meta-parameter updating method for traditional meta-learning has been adjusted, enabling the meta-parameters to attain a better position. By doing so, we can rapidly improve the generalization and convergence of the meta-learning parameters on unknown tasks. Experimental results indicate a 2.1% improvement over the base model in the 4-conv setting, with a more pronounced effect as the neural network is progressively complexified, reaching a 4.1% improvement in resnet12.