A Survey of Multilingual Neural Machine Translation Based on Sparse Models

Abstract

Recent research has shown a burgeoning interest in exploring sparse models for massively Multilingual Neural Machine Translation (MNMT). In this paper, we present a comprehensive survey of this emerging topic. Massively MNMT, when based on sparse models, offers significant improvements in parameter efficiency and reduces interference compared to its dense model counterparts. Various methods have been proposed to leverage sparse models for enhancing translation quality. However, the lack of a thorough survey has hindered the identification and further investigation of the most promising approaches. To address this gap, we provide an exhaustive examination of the current research landscape in massively MNMT, with a special emphasis on sparse models. Initially, we categorize the various sparse model-based approaches into distinct classifications. We then delve into each category in detail, elucidating their fundamental modeling principles, core issues, and the challenges they face. Wherever possible, we conduct comparative analyses to assess the strengths and weaknesses of different methodologies. Moreover, we explore potential future research avenues for MNMT based on sparse models. This survey serves as a valuable resource for both newcomers and established experts in the field of MNMT, particularly those interested in sparse model applications.