LDM-KGC: A low-dimensional knowledge graph completion model based on multi-head attention mechanism

Abstract

Existing Transformer-based knowledge graph completion methods often rely on high-dimensional embeddings to achieve competitive performance, which to some extent limits their scalability on large-scale knowledge graphs. To address this challenge, the LDM-KGC model based on the multi-head attention mechanism is proposed. By combining QKV-layer and Update-layer, LDM-KGC can not only learn rich information but also reduce information loss during training, thereby achieving superior embedding representations in low-dimensional spaces. Specifically, QKV-layer utilizes the multi-head attention mechanism to effectively capture interactions between entities and relations, while Update-layer further refines the resulting embeddings. Experimental results on the FB15k-237 and WN18RR datasets demonstrate that LDM-KGC outperforms 14 baseline models, significantly improving mean reciprocal rank (MRR) by 12.4 percentage points and 24.4 percentage points over the worst baseline, respectively. Notably, LDM-KGC achieves MRR of 36.5%, Hits@1 of 27.1%, Hits@3 of 40.2%, and Hits@10 of 55.2% on the FB15k-237 dataset. Furthermore, LDM-KGC reaches a Hits@10 score of 65.2% on the NELL-995 dataset. These results underscore the effectiveness of LDM-KGC in generating low-dimensional embeddings, thereby offering a scalable solution for large-scale knowledge graph completion.