多语言表面实现的迁移学习语法

Atif Khurshid, Seemab Latif, R. Latif
{"title":"多语言表面实现的迁移学习语法","authors":"Atif Khurshid, Seemab Latif, R. Latif","doi":"10.1109/ICoDT252288.2021.9441522","DOIUrl":null,"url":null,"abstract":"Deep learning approaches to surface realisation are often held back by the lack of good quality datasets. These datasets require significant human effort to design and are rarely available for low-resource languages. We investigate the possibility of cross-lingual transfer learning of grammatical features in a multilingual text-to-text transformer. We train several mT5-small transformer models to generate grammatically correct sentences by reordering and inflecting words, first using monolingual data in one language and then in another language. We show that language comprehension and task-specific performance of the models benefit from pretraining on other languages with similar grammar rules, while languages with dissimilar grammar appear to disorient the model from its previous training. The results indicate that a model trained on multiple languages may familiarize itself with their common features and, thus, require less data and processing time for language-specific training. However, the experimental models are limited by their entirely text-to-text approach and insufficient computational power. A complete multilingual realisation model will require a more complex transformer variant and longer training on more data.","PeriodicalId":207832,"journal":{"name":"2021 International Conference on Digital Futures and Transformative Technologies (ICoDT2)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2021-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Transfer Learning Grammar for Multilingual Surface Realisation\",\"authors\":\"Atif Khurshid, Seemab Latif, R. Latif\",\"doi\":\"10.1109/ICoDT252288.2021.9441522\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Deep learning approaches to surface realisation are often held back by the lack of good quality datasets. These datasets require significant human effort to design and are rarely available for low-resource languages. We investigate the possibility of cross-lingual transfer learning of grammatical features in a multilingual text-to-text transformer. We train several mT5-small transformer models to generate grammatically correct sentences by reordering and inflecting words, first using monolingual data in one language and then in another language. We show that language comprehension and task-specific performance of the models benefit from pretraining on other languages with similar grammar rules, while languages with dissimilar grammar appear to disorient the model from its previous training. The results indicate that a model trained on multiple languages may familiarize itself with their common features and, thus, require less data and processing time for language-specific training. However, the experimental models are limited by their entirely text-to-text approach and insufficient computational power. A complete multilingual realisation model will require a more complex transformer variant and longer training on more data.\",\"PeriodicalId\":207832,\"journal\":{\"name\":\"2021 International Conference on Digital Futures and Transformative Technologies (ICoDT2)\",\"volume\":\"8 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2021-05-20\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2021 International Conference on Digital Futures and Transformative Technologies (ICoDT2)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/ICoDT252288.2021.9441522\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2021 International Conference on Digital Futures and Transformative Technologies (ICoDT2)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICoDT252288.2021.9441522","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

摘要

深度学习的表面实现方法经常受到缺乏高质量数据集的阻碍。这些数据集需要大量的人力来设计,并且很少用于资源匮乏的语言。我们研究了在多语言文本到文本转换器中语法特征的跨语言迁移学习的可能性。我们首先使用一种语言的单语言数据,然后再使用另一种语言的单语言数据,训练了几个mT5-small transformer模型,通过重新排序和屈折单词来生成语法正确的句子。我们表明,模型的语言理解和任务特定性能受益于对具有相似语法规则的其他语言的预训练,而具有不同语法规则的语言似乎会使模型从先前的训练中迷失方向。结果表明,在多种语言上训练的模型可以熟悉它们的共同特征,从而减少特定语言训练所需的数据和处理时间。然而,实验模型受到完全文本到文本方法和计算能力不足的限制。一个完整的多语言实现模型将需要一个更复杂的转换器变体和更多数据上的更长时间的训练。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Transfer Learning Grammar for Multilingual Surface Realisation
Deep learning approaches to surface realisation are often held back by the lack of good quality datasets. These datasets require significant human effort to design and are rarely available for low-resource languages. We investigate the possibility of cross-lingual transfer learning of grammatical features in a multilingual text-to-text transformer. We train several mT5-small transformer models to generate grammatically correct sentences by reordering and inflecting words, first using monolingual data in one language and then in another language. We show that language comprehension and task-specific performance of the models benefit from pretraining on other languages with similar grammar rules, while languages with dissimilar grammar appear to disorient the model from its previous training. The results indicate that a model trained on multiple languages may familiarize itself with their common features and, thus, require less data and processing time for language-specific training. However, the experimental models are limited by their entirely text-to-text approach and insufficient computational power. A complete multilingual realisation model will require a more complex transformer variant and longer training on more data.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信