Bilingual Text Classification in English and Indonesian via Transfer Learning using XLM-RoBERTa

Q3 Computer Science

International Journal of Advances in Soft Computing and its Applications Pub Date : 2021-11-28 DOI:10.15849/ijasca.211128.06

Yakobus Wiciaputra, J. Young, A. Rusli

{"title":"Bilingual Text Classification in English and Indonesian via Transfer Learning using XLM-RoBERTa","authors":"Yakobus Wiciaputra, J. Young, A. Rusli","doi":"10.15849/ijasca.211128.06","DOIUrl":null,"url":null,"abstract":"With the large amount of text information circulating on the internet, there is a need of a solution that can help processing data in the form of text for various purposes. In Indonesia, text information circulating on the internet generally uses 2 languages, English and Indonesian. This research focuses in building a model that is able to classify text in more than one language, or also commonly known as multilingual text classification. The multilingual text classification will use the XLM-RoBERTa model in its implementation. This study applied the transfer learning concept used by XLM-RoBERTa to build a classification model for texts in Indonesian using only the English News Dataset as a training dataset with Matthew Correlation Coefficient value of 42.2%. The results of this study also have the highest accuracy value when tested on a large English News Dataset (37,886) with Matthew Correlation Coefficient value of 90.8%, accuracy of 93.3%, precision of 93.4%, recall of 93.3%, and F1 of 93.3% and the accuracy value when tested on a large Indonesian News Dataset (70,304) with Matthew Correlation Coefficient value of 86.4%, accuracy, precision, recall, and F1 values of 90.2% using the large size Mixed News Dataset (108,190) in the model training process. Keywords: Multilingual Text Classification, Natural Language Processing, News Dataset, Transfer Learning, XLM-RoBERTa","PeriodicalId":38638,"journal":{"name":"International Journal of Advances in Soft Computing and its Applications","volume":" ","pages":""},"PeriodicalIF":0.0000,"publicationDate":"2021-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Advances in Soft Computing and its Applications","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.15849/ijasca.211128.06","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"Computer Science","Score":null,"Total":0}

引用次数: 1

Abstract

With the large amount of text information circulating on the internet, there is a need of a solution that can help processing data in the form of text for various purposes. In Indonesia, text information circulating on the internet generally uses 2 languages, English and Indonesian. This research focuses in building a model that is able to classify text in more than one language, or also commonly known as multilingual text classification. The multilingual text classification will use the XLM-RoBERTa model in its implementation. This study applied the transfer learning concept used by XLM-RoBERTa to build a classification model for texts in Indonesian using only the English News Dataset as a training dataset with Matthew Correlation Coefficient value of 42.2%. The results of this study also have the highest accuracy value when tested on a large English News Dataset (37,886) with Matthew Correlation Coefficient value of 90.8%, accuracy of 93.3%, precision of 93.4%, recall of 93.3%, and F1 of 93.3% and the accuracy value when tested on a large Indonesian News Dataset (70,304) with Matthew Correlation Coefficient value of 86.4%, accuracy, precision, recall, and F1 values of 90.2% using the large size Mixed News Dataset (108,190) in the model training process. Keywords: Multilingual Text Classification, Natural Language Processing, News Dataset, Transfer Learning, XLM-RoBERTa

查看原文本刊更多论文

基于迁移学习的英语和印尼语双语文本分类

随着大量文本信息在互联网上传播，需要一种解决方案来帮助处理各种目的的文本形式的数据。在印度尼西亚，互联网上流传的文本信息通常使用两种语言，英语和印尼语。这项研究的重点是建立一个能够对多种语言的文本进行分类的模型，或者通常称为多语言文本分类。多语言文本分类将在实施中使用XLM-RoBERTa模型。本研究应用XLM RoBERTa使用的迁移学习概念，仅使用英语新闻数据集作为训练数据集，建立了印尼语文本的分类模型，Matthew相关系数为42.2%。在大型英语新闻数据集中（37886）进行测试时，本研究的结果也具有最高的准确性值，Matthew相关性系数为90.8%，准确率93.3%，准确度93.4%，召回率93.3%和F1为93.3%，以及在模型训练过程中使用大型混合新闻数据集（108190）在Matthew相关系数值为86.4%的大型印尼新闻数据集上测试时的准确度值（70304），准确度、准确度、召回率和F1值为90.2%。关键词：多语言文本分类，自然语言处理，新闻数据集，迁移学习，XLM-RoBERTa

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

International Journal of Advances in Soft Computing and its Applications Computer Science-Computer Science Applications

CiteScore

3.30

自引率

0.00%

发文量

期刊介绍： The aim of this journal is to provide a lively forum for the communication of original research papers and timely review articles on Advances in Soft Computing and Its Applications. IJASCA will publish only articles of the highest quality. Submissions will be evaluated on their originality and significance. IJASCA invites submissions in all areas of Soft Computing and Its Applications. The scope of the journal includes, but is not limited to: √ Soft Computing Fundamental and Optimization √ Soft Computing for Big Data Era √ GPU Computing for Machine Learning √ Soft Computing Modeling for Perception and Spiritual Intelligence √ Soft Computing and Agents Technology √ Soft Computing in Computer Graphics √ Soft Computing and Pattern Recognition √ Soft Computing in Biomimetic Pattern Recognition √ Data mining for Social Network Data √ Spatial Data Mining & Information Retrieval √ Intelligent Software Agent Systems and Architectures √ Advanced Soft Computing and Multi-Objective Evolutionary Computation √ Perception-Based Intelligent Decision Systems √ Spiritual-Based Intelligent Systems √ Soft Computing in Industry ApplicationsOther issues related to the Advances of Soft Computing in various applications.