Pseudo-kNN-MT: Enhancing domain adaptability of neural machine translation via target language data

Abstract

Although Neural Machine Translation (NMT) has recently achieved remarkable performance improvements, it still faces challenges in domain adaptation. Previous research has focused on mitigating this issue by integrating translation knowledge from bilingual domain data. However, the limited availability of bilingual translation resources has constrained these methods in real world application. To address this inadequacy, solutions based on monolingual data, such as back-translation, have been proposed. Nevertheless, these methods often incur additional training costs due to the necessity of training reverse models to generate pseudo data. In light of this, we propose Pseudo-$k$NN-MT, which does not require additional training. This method creates pseudo-bilingual data pairs by retrieving semantically similar sentences from target language data and subsequently builds the $k$NN datastore. To effectively reduce the noise introduced by the pseudo-data, we incorporate cross-lingual retrieval distances into the $k$NN probability construction process. Experiments in both high-resource and low-resource machine translation scenarios across multiple domains demonstrate that our method significantly improves the domain adaptation capabilities of NMT in both settings, yielding average improvements of 6.08 and 7.70 SacreBLEU points and 0.66 and 1.62 COMET scores on the multi-domain dataset, respectively.