VarDial@COLING 2018最新文献

筛选

英文中文

Encoder-Decoder Methods for Text Normalization 文本规范化的编码器-解码器方法

VarDial@COLING 2018 Pub Date : 2018-08-20 DOI: 10.5167/UZH-156775

M. Lusetti, T. Ruzsics, A. Göhring, T. Samardžić, E. Stark

{"title":"Encoder-Decoder Methods for Text Normalization","authors":"M. Lusetti, T. Ruzsics, A. Göhring, T. Samardžić, E. Stark","doi":"10.5167/UZH-156775","DOIUrl":"https://doi.org/10.5167/UZH-156775","url":null,"abstract":"Text normalization is the task of mapping non-canonical language, typical of speech transcription and computer-mediated communication, to a standardized writing. It is an up-stream task necessary to enable the subsequent direct employment of standard natural language processing tools and indispensable for languages such as Swiss German, with strong regional variation and no written standard. Text normalization has been addressed with a variety of methods, most successfully with character-level statistical machine translation (CSMT). In the meantime, machine translation has changed and the new methods, known as neural encoder-decoder (ED) models, resulted in remarkable improvements. Text normalization, however, has not yet followed. A number of neural methods have been tried, but CSMT remains the state-of-the-art. In this work, we normalize Swiss German WhatsApp messages using the ED framework. We exploit the flexibility of this framework, which allows us to learn from the same training data in different ways. In particular, we modify the decoding stage of a plain ED model to include target-side language models operating at different levels of granularity: characters and words. Our systematic comparison shows that our approach results in an improvement over the CSMT state-of-the-art.","PeriodicalId":431809,"journal":{"name":"VarDial@COLING 2018","volume":"120 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123231151","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 38

Twist Bytes - German Dialect Identification with Data Mining Optimization 扭曲字节-德语方言识别与数据挖掘优化

VarDial@COLING 2018 Pub Date : 2018-08-01 DOI: 10.21256/ZHAW-4850

F. Souza, Ralf Grubenmann, Pius von Däniken, Dirk Von Gruenigen, Jan Deriu, Mark Cieliebak

引用次数: 9