MKE-PLLM: A benchmark for multilingual knowledge editing on pretrained large language model

Multilingual large language models have demonstrated remarkable performance across various downstream tasks but are still plagued by factuality errors. Knowledge editing aims to correct these errors by modifying the internal knowledge of pre-trained models. However, current knowledge editing methods primarily focus on monolingual settings, neglecting the complexities and interdependencies within multilingual scenarios. Furthermore, benchmarks specifically designed for multilingual knowledge editing are relatively scarce. Addressing this gap, this paper constructs a novel multilingual knowledge editing benchmark. This benchmark comprehensively evaluates methods for mLLMs based on accuracy, reliability, generalization, and consistency. To ensure the robustness and usability of the benchmark, we conducted detailed analysis and validation. Concurrently, we propose a baseline method that adapts existing monolingual knowledge editing techniques to the multilingual environment. Extensive experimental results demonstrate the effectiveness of our constructed benchmark in evaluating multilingual knowledge editing.