Imitation from Observation using RL and Graph-based Representation of Demonstrations

Abstract

Teaching robots behavioral skills by leveraging examples provided by an expert, also referred to as Imitation Learning from Observation (IfO or ILO), is a promising approach for learning novel tasks without requiring a task-specific reward function to be engineered. We propose a RL-based framework to teach robots manipulation tasks given expert observation-only demonstrations. First, a representation model is trained to extract spatial and temporal features from demonstrations. Graph Neural Networks (GNNs) are used to encode spatial patterns, while LSTMs and Transformers are used to encode temporal features. Second, based on an off-the-shelf RL algorithm, the demonstrations are leveraged through the trained representation to guide the policy training towards solving the task demonstrated by the expert. We show that our approach compares favorably to state-of-the-art IfO algorithms with a 99% success rate and transfers well to the real world.