A knowledge-fused maximum mean discrepancy for cross-lingual named entity recognition

Abstract

Cross-lingual named entity recognition (NER) aims to train a model that effectively transfers knowledge from a source language to a target language using labeled data from the source. This approach addresses the challenges posed by the limited availability of NER resources in certain languages. In such transfer learning tasks, the Maximum Mean Discrepancy (MMD) loss function is commonly used to minimize the discrepancy between the source and target domains. However, computing the MMD loss is computationally intensive. Traditional methods often use sampling methods for approximate calculations. But from an accuracy perspective, sampling without prior knowledge yields suboptimal results. To address these challenges, we fuse part-of-speech knowledge into the computation of MMD. Specifically, we replace words of various parts of speech in the sentence with [MASK] token at a specific proportion. We then obtain category labels based on the part of speech of the replaced words. Subsequently, we perform stratified sampling based on these category labels to achieve more accurate results in the MMD calculation. Experiments on multiple benchmark datasets show that our model outperforms existing methods.