CTPR: Contrastive transition predictive representation for reinforcement learning

Abstract

Learning policies from high-dimensional observations is a challenging problem for pixel-based reinforcement learning. Most existing pixel-based reinforcement learning methods struggle with the inefficiency of extracting meaningful state representations from raw pixel data, lacking temporal correlation and resulting in suboptimal performance. To this end, we propose a innovative method named contrastive transition predictive representation for reinforcement learning (CTPR), which utilizes contrastive learning and a transition model to efficiently extract high-level state representations from raw pixels for sample-efficient reinforcement learning. In the reinforcement learning component, we perform policy control based on the learned contrastive representations. We have evaluated the effectiveness of the proposed method by conducting numerous experiments on DeepMind Control, and the results show that our method has achieve significant improvements over the state-of-the-art methods.