Semi-supervised learning in a complex arm motor control task

Abstract

In real world learning problems it is often the case that while the amount of labeled training data is limited, the amount of raw, unlabeled data available is vast. It is thus beneficial to develop ways of exploiting the large amount of unlabeled data to maximize the utility of each labeled sample. We examine this “semi-supervised” learning problem in the context of a flexible arm with complex dynamics. The goal of the learning process is to predict a reward value R, which evaluates the system's performance on a given task, from an input motor command M. We assume that the number of trials for which the reward is given is strictly limited. This makes it difficult to learn the function M → R, because of the complex dynamics of the arm. We also assume that there are a large number of unsupervised trials which give information about the trajectory I that results from a particular motor command M. Our method is to first learn a mapping from the motor command M to the trajectory I from the unsupervised samples, and then learn a mapping from I to the reward value R from the supervised samples. We show that the indirect learning process M → I → R achieves superior performance to the direct process M → R, under a wide variety of conditions.