Diachronic semantic encoding based on pre-trained language model for temporal knowledge graph reasoning

Abstract

Temporal Knowledge Graph Reasoning (TKGR) aims to infer missing facts at specific timestamps. However, most existing methods primarily focus on the local and global evolutionary characteristics of temporal knowledge graphs (TKG), often neglecting the inherent semantic information of historical facts. The oversight limits the understanding of the diachronic evolution of facts, thereby limiting the ability to predict future missing facts. To address these issues, we propose a TKGR model with Diachronic Semantic Encoding based on a Pre-trained language model, called DSEP. It uses a pre-trained language model (PLM) to learn the evolutionary characteristics of historical related facts of the entity or relation to be predicted, so as to enhance the understanding of historical facts by the graph encoder used to capture the local evolutionary characteristics of the temporal knowledge graph. Additionally, to further narrow the prediction scope, DSEP incorporates historical fact correlation matrix in its prediction results. Experimental results on four benchmark datasets demonstrate that DSEP significantly improves the performance of relation prediction in temporal knowledge graphs, with an average improvement of 20.9% in MRR¹.