Reinforcement knowledge graph reasoning based on dual agents and attention mechanism

Abstract

Reinforcement learning can model knowledge graph multi-hop reasoning as Markov Decision Processes and improve the accuracy and interpretability of predicting paths between entities. Existing reasoning methods usually ignore the logic of action selection when facing one-to-many or many-to-many relationships, resulting in poor performance in knowledge graph reasoning. Furthermore, the general multi-hop reasoning only achieves effective short-path reasoning and lacks efficiency in long-distance reasoning. To address the above challenges, we propose a reinforcement learning reasoning model based on dual agents and attention mechanism, where two agents are trained at the macro and micro levels, and the macro agent guides the reasoning of the micro agent. The model employs an attention mechanism to enhance the representation of the current state of the agent, to help the policy network in making more appropriate action selections when facing one-to-many or many-to-many relationships, so as to improve the selection efficiency. Simultaneously, we propose a reward function with a penalty mechanism that penalizes the agent for prematurely reaching the correct answer without staying in place, and enhances the reward of the micro agent with the reward of the macro agent. The two agents cooperate with each other to find reasoning paths on the knowledge graph. Finally, we compare the proposed model with six well-known inference method baselines on three benchmark datasets, and the experimental results show that our proposed method achieves very competitive results.