Reinforced Heterogeneous Graphlet Design for Knowledge Graph Representation Learning

Knowledge graphs (KGs) are practical tools that represent and integrate plentiful structural and semantic information in mainstream industrial scenarios. Despite their potential, the heterogeneity and complexity of KGs pose a formidable obstacle, especially for graph representation learning. Most existing KG embedding models omit dynamic high-order connectivity patterns to gain insights into heterogeneous networks and heavily rely on handcrafted patterns to handle complex semantic relationships, which limits their capability to adaptively capture the nuanced and intricate relationships of KGs in different tasks. To fill this gap, we present Reinforced Heterogeneous Graphlet Design (ReHGD)—a model designed for KGs that focuses on the adaptive design of typed graphlets (heterogeneous chains and motifs) through a cooperative multi-agent reinforcement learning algorithm. This task-driven approach can learn discriminative graph representations tailored to specific downstream tasks. Specifically, ReHGD engages in the creation of typed graphlets through a two-stage process: it (1) establishes a reinforced chain design module to generate chains without predefined rules and (2) employs a buffer-aware sampling technique to derive episodic chains from prior experiences. Subsequently, motifs are deduced through the application of commute count and Hadamard product operations to the episodic chain-based subgraphs. In the final step toward learning graph representations, ReHGD undertakes chain and motif aggregations. Experimental results and analyses reveal that ReHGD outperforms strong baselines on three real-world graph data and practical tasks.