Enhancing Reinforcement Learning With Cross-Domain Knowledge Transfer via Seeded Graph Matching.

Abstract

Transfer reinforcement learning (TRL) aims to boost the efficiency of reinforcement learning (RL) agents by leveraging knowledge from related tasks. Prior research primarily focuses on intradomain transfer, overlooking the complexities of transferring knowledge across tasks with differing state and action spaces. Recent efforts in cross-domain TRL aim to bridge this gap by establishing mappings between disparate source and target spaces, thereby enabling knowledge transfer across RL tasks with varied state and action configurations. However, existing studies often rely on strict prior assumptions about the relationships between state spaces, which limits their practical generality. In this article, we propose a novel approach to cross-domain TRL based on seeded graph matching, which enables alignment between source and target tasks regardless of differences in their state-action spaces. In particular, we model RL tasks as directed graphs, identify seed node pairs based on common RL properties, and devise a graph matching algorithm to align the source and target tasks by leveraging their structural characteristics. Building on this alignment, we introduce a policy-based transfer algorithm that improves the performance of the target RL task as its RL process progresses. Finally, we conduct comprehensive empirical studies on both discrete and continuous tasks with diverse state-action spaces. The experimental results validate the effectiveness of the proposed algorithm.