Using binary classification to evaluate the quality of machine translators

Abstract

Machine translators have become increasingly popular and currently play an important role because of their great assistance in cross-cultural communication. However, machine translators often produces some unnatural texts, and an evaluation of machine translators is thus needed to avoid the abuse of machine-translated texts. This paper presents the use of binary classification to evaluate the quality of machine translators without references. First, we construct a large-scale dataset including humangenerated texts and machine-translated texts. Second, the dataset is used to train the multiple binary classifiers, e.g., decision tree, random forest, extreme gradient boosting, support vector machines, logistic regression, etc. Finally, these trained classifiers constitute the ensemble model by majority voting, and this ensemble model is used to evaluate the qualities of machine-translated texts. Experimental results show that the proposed evaluation method better measures the qualities of translated texts by some commercial machine translators.