Leveraging CNN and Bi-LSTM in Indonesian G2P Using Transformer

We apply a transformer called tensor2tensor toolkit, which is based on Tensorflow, to overcome the Grapheme-to-Phoneme conversion problem. This study performs conversions to produce pronunciation symbols for certain letter sequences in Indonesian particularly. The unavailability of the G2P conversion system in Indonesian is currently being faced, so research is being carried out to create a system that can solve this problem by applying the Transformer. The transformer has a simple network architecture based solely on the attention mechanism, so we took advantage of eliminating convolution and redundancies—complex recurrent and convolution neural networks including encoders and decoders as the basis for the sequence transduction model. The excellent performance of the model is obtained through the attention mechanism by connecting the encoder and decoder. By using this tool, we carry out to compare among KBBI and CMU dictionary datasets. We attained a word error rate (WER) of 6,7% on the KBBI data set after training for three days on two core CPUs, which has an accuracy of 93,3%, improving over the existing best results CMU dictionary dataset for 26% word error rate. In this study, we carried out a detailed experimental evaluation by assessing the processing time and the error rate of words and then compared it with state of the art. By demonstrating this Transformer, this tool successfully generalizes and then applies it to several Indonesian elements with limited training data and large training data. We concluded that the transformer model is suitable for dealing with the G2P problem at hand for this task.