Alignment Offset Based Adaptive Training for Simultaneous Machine Translation

Given incomplete source sentences as inputs, it is generally difficult for Simultaneous Machine Translation (SiMT) models to generate a target token once its aligned source tokens are absent. How to measure such difficulty and further conduct adaptive training for SiMT models are not sufficiently studied. In this paper, we propose a new metric named alignment offset (AO) to quantify the learning difficulty of target tokens for SiMT models. Given a target token, its AO is calculated by the offset between its aligned source tokens and the already received source tokens. Furthermore, we design two AO-based adaptive training methods to improve the training of SiMT models. Firstly, we introduce token-level curriculum learning based on AO, which progressively switches the training process from easy target tokens to difficult ones. Secondly, we assign an appropriate weight to the training loss of each target token according to its AO. Experimental results on four datasets demonstrate that our methods significantly and consistently outperform all the strong baselines.