Script event prediction based on pre-trained model with tail event enhancement

Abstract

Script event prediction is a big challenge and its goal is to predict the subsequent event based on the observed events. Since an event is described by text, the pre-trained models have been applied for event representation. However, the embedding based on the pre-trained models is sensitive to the short text format of events, and the existing works do not handle it well. In addition, previous models pay more attention to the semantic similarity but ignore the factors of emergencies. The turning event at the tail of the event chain can easily affect the follow-up direction. This paper proposes a new preprocessing method: cleaning, alignment, and connection, which helps to obtain richer event representations. On this basis, we concatenate the embedding of the CLS token and event sequence to integrate the semantic and temporal features of the event chain. To deal with the problem of event turning, we propose a tail event enhancement module. It adds the transition probability of tail events and candidate events into prediction layer, so as to avoid pay only attention to the semantic feature. The results of a large number of comparative experiments and ablation experiments confirm the superiority of our model compared with the baselines.