Linear self-attention with multi-relational graph for knowledge graph completion

Abstract

Knowledge graph completion (KGC) aims to infer missing facts based on the existing knowledge. Graph Convolutional Networks (GCNs) have gained significant traction due to their proficiency in effectively modeling graph structures, especially within the realm of Knowledge Graph Completion (KGC). In GCN-based KGC methodologies, GCNs are initially employed to generate comprehensive representations of entities, followed by the application of Knowledge Graph Embedding (KGE) models to elucidate the interactions among entities and relations. However, most GCN-based KGC models ignore the long-range pairwise relationships in the graph. To address these limitations and enhance KGC, we propose a model called Linear Self-Attention with Multi-Relational Graph Network (LTRGN). Specifically, this model merges GCN and linear self-attention to serve as the encoder. This model introduces a linear self-attention that can capture long-range node dependencies without introducing excessive computational overhead. Furthermore, we implement an attention mechanism designed to better assess the significance of various neighboring nodes relative to the source node. We demonstrate the effectiveness of the proposed LTRGN on the standard FB15k-237, WN18RR, Kinship, and UMLS datasets. On the dense graphs Kinship and UMLS, the MRR of our model improves by 1.3% and 4.1%, respectively, while Hits@1 increases by 1.7% and 6.4% compared to the best-performing model. The results show the efficacy of the model for the KGC task. The code is released at https://github.com/lixianqingliuyan/LTRGN.