Fuzzy-based Text Augmentation to boost the Statistical Machine Translation for Indic Languages

Artificial intelligence technology has been used in various industries to give convenience to people's lives. However, real-world applications often face a critical challenge of data scarcity. Text Augmentation (TA) techniques are being investigated extensively in the field of natural language processing to solve this data scarcity and enhance model performance. For Indian Languages, data collection is challenging as they exhibit rich syntactic and morphological diversity compared to resource-rich languages like English. This diversity further compounds the problem of data scarcity, leading to poor translation quality, especially when translating from low-resource languages to resource-rich ones. This study addresses the challenge by proposing a fuzzy-based TA technique to enhance machine translation quality. The proposed approach leverages fuzzy matching to identify and utilize potential near-matches in translated sentences, thereby augmenting the available training data. Fuzzy is a lexicalized matching strategy that seeks out non-exact matches in a sentence. To evaluate the effectiveness of this method, three resource-rich Indic languages were considered, including a low-resource endangered language. Experimental results on the test set demonstrate significant and consistent improvements in the augmented dataset, achieving a +3.53 of BiLingual Evaluation Understudy (BLEU) and +6.247 of Metric for Evaluation of Translation with Explicit ORdering (METEOR) point increase over the baseline system. Furthermore, we conducted statistical analysis to confirm the significance of these results, validating the enhanced quality of the translation tasks.