Exploring deep reinforcement learning acceleration by superscaling data augmentation via branched fractal symmetries.

Abstract

Learning deep reinforcement learning (DRL) policies directly in physical robots remains bottlenecked by slow wall-clock training times. We present preliminary research on Branched Euclidean Group Fractal Symmetries, a trajectory-level augmentation framework that super-scales group transformations to accelerate policy learning for manipulation. We model a Markov decision process (MDP) as a tree of state-action pairs; at each depth, affine transformations generate geometric structures, within which combinations of Euclidean group symmetries produce a large set of unique symmetric equivalences. A self-similar branching rule can also be iterated across depths to yield fractal symmetry expansions that populate the replay buffer with diverse yet consistent experiences with Cartesian or hand-centered image representations. We show the efficacy of our approach through simulated and real Franka robot manipulation contact tasks. Our study learns robust policies directly in physical robots in as fast as 14 min and in 20 min of wall-clock time on average. The code is accessible at https://github.com/lippyRobotics/fractalserl.