A hierarchical and interlamination graph self-attention mechanism-based knowledge graph reasoning architecture

Abstract

Knowledge Graph (KG) is an essential research field in graph theory, but its inherent incompleteness and sparsity influence its performance in several fields. Knowledge Graph Reasoning (KGR) aims to ameliorate those problems by mining new knowledge from subsistent knowledge. As one of the downstream tasks of KGR, link prediction is of great significance for improving the quality of KG. Recently, the Graph Neural Network (GNN)-based method became the most effective way to achieve the link prediction task. However, it still suffers from problems such as incomplete neighbor and relation-level information aggregation and unstable learning of the entity's features. To improve those issues, a Hierarchical and Interlamination Graph Self-attention Mechanism-based (HIGSM) plug-and-play architecture is proposed for KGR in this paper. It is composed of three-level layers: feature extractor, encoder, and decoder. The feature extractor makes our architecture more effective and stable for the retrieval of new features. The encoder is equipped with a two-stage encoding mechanism accompanied by two mixture-of-expert strategies, which enables our architecture to capture more practical reasoning information to improve prediction accuracy and generalization of the model. The decoder can use existing KGR models and compute the scores of triples in KG. The extensive experimental results and ablation studies on four KGs unambiguously demonstrate the state-of-the-art prediction performance of the proposed HIGSM architecture compared to current GNN-based methods.