Comparing Machine Translation Accuracy of Attention Models

Abstract

Machine translation models using encoder and decoder architecture do not give accuracy as high as expectation. One reason for this ineffectiveness is due to lack of attention mechanism during training phase. Attention-based models overcome drawbacks of previous ones and obtain noteworthy improvement in terms of accuracy. In the paper, we experiment three attention models and evaluate their BLEU scores on small data sets. Bahdanau model achieves high accuracy, Transformer model obtains good accuracy while Luong model only gets acceptable accuracy.