International Conference on Language Resources and Evaluation最新文献_第3页

What a Creole Wants, What a Creole Needs 克里奥尔人想要什么，克里奥尔人需要什么

International Conference on Language Resources and Evaluation Pub Date : 2022-06-01 DOI: 10.48550/arXiv.2206.00437

Heather Lent, Kelechi Ogueji, Miryam de Lhoneux, Orevaoghene Ahia, Anders Søgaard

引用次数: 9

Hollywood Identity Bias Dataset: A Context Oriented Bias Analysis of Movie Dialogues 好莱坞身份偏见数据集:电影对白的语境导向偏见分析

International Conference on Language Resources and Evaluation Pub Date : 2022-05-31 DOI: 10.48550/arXiv.2205.15951

Sandhya Singh, Prapti Roy, Nihar Sahoo, Niteesh Mallela, Himanshu Gupta, P. Bhattacharyya, Milind Savagaonkar, Nidhi, Roshni Ramnani, Anutosh Maitra, Shubhashis Sengupta

{"title":"Hollywood Identity Bias Dataset: A Context Oriented Bias Analysis of Movie Dialogues","authors":"Sandhya Singh, Prapti Roy, Nihar Sahoo, Niteesh Mallela, Himanshu Gupta, P. Bhattacharyya, Milind Savagaonkar, Nidhi, Roshni Ramnani, Anutosh Maitra, Shubhashis Sengupta","doi":"10.48550/arXiv.2205.15951","DOIUrl":"https://doi.org/10.48550/arXiv.2205.15951","url":null,"abstract":"Movies reflect society and also hold power to transform opinions. Social biases and stereotypes present in movies can cause extensive damage due to their reach. These biases are not always found to be the need of storyline but can creep in as the author’s bias. Movie production houses would prefer to ascertain that the bias present in a script is the story’s demand. Today, when deep learning models can give human-level accuracy in multiple tasks, having an AI solution to identify the biases present in the script at the writing stage can help them avoid the inconvenience of stalled release, lawsuits, etc. Since AI solutions are data intensive and there exists no domain specific data to address the problem of biases in scripts, we introduce a new dataset of movie scripts that are annotated for identity bias. The dataset contains dialogue turns annotated for (i) bias labels for seven categories, viz., gender, race/ethnicity, religion, age, occupation, LGBTQ, and other, which contains biases like body shaming, personality bias, etc. (ii) labels for sensitivity, stereotype, sentiment, emotion, emotion intensity, (iii) all labels annotated with context awareness, (iv) target groups and reason for bias labels and (v) expert-driven group-validation process for high quality annotations. We also report various baseline performances for bias identification and category detection on our dataset.","PeriodicalId":260442,"journal":{"name":"International Conference on Language Resources and Evaluation","volume":"192 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124269964","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

Context-based Virtual Adversarial Training for Text Classification with Noisy Labels 基于上下文的带噪声标签文本分类虚拟对抗训练

International Conference on Language Resources and Evaluation Pub Date : 2022-05-29 DOI: 10.48550/arXiv.2206.11851

Do-Myoung Lee, Yeachan Kim, Chang-gyun Seo

{"title":"Context-based Virtual Adversarial Training for Text Classification with Noisy Labels","authors":"Do-Myoung Lee, Yeachan Kim, Chang-gyun Seo","doi":"10.48550/arXiv.2206.11851","DOIUrl":"https://doi.org/10.48550/arXiv.2206.11851","url":null,"abstract":"Deep neural networks (DNNs) have a high capacity to completely memorize noisy labels given sufficient training time, and its memorization unfortunately leads to performance degradation. Recently, virtual adversarial training (VAT) attracts attention as it could further improve the generalization of DNNs in semi-supervised learning. The driving force behind VAT is to prevent the models from overffiting to data points by enforcing consistency between the inputs and the perturbed inputs. These strategy could be helpful in learning from noisy labels if it prevents neural models from learning noisy samples while encouraging the models to generalize clean samples. In this paper, we propose context-based virtual adversarial training (ConVAT) to prevent a text classifier from overfitting to noisy labels. Unlike the previous works, the proposed method performs the adversarial training in the context level rather than the inputs. It makes the classifier not only learn its label but also its contextual neighbors, which alleviate the learning from noisy labels by preserving contextual semantics on each data point. We conduct extensive experiments on four text classification datasets with two types of label noises. Comprehensive experimental results clearly show that the proposed method works quite well even with extremely noisy settings.","PeriodicalId":260442,"journal":{"name":"International Conference on Language Resources and Evaluation","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133509862","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Merkel Podcast Corpus: A Multimodal Dataset Compiled from 16 Years of Angela Merkel’s Weekly Video Podcasts 默克尔播客语料库:从默克尔16年的每周视频播客中编译的多模态数据集

International Conference on Language Resources and Evaluation Pub Date : 2022-05-24 DOI: 10.48550/arXiv.2205.12194

Debjoy Saha, Shravan Nayak, Timo Baumann

{"title":"Merkel Podcast Corpus: A Multimodal Dataset Compiled from 16 Years of Angela Merkel’s Weekly Video Podcasts","authors":"Debjoy Saha, Shravan Nayak, Timo Baumann","doi":"10.48550/arXiv.2205.12194","DOIUrl":"https://doi.org/10.48550/arXiv.2205.12194","url":null,"abstract":"We introduce the Merkel Podcast Corpus, an audio-visual-text corpus in German collected from 16 years of (almost) weekly Internet podcasts of former German chancellor Angela Merkel. To the best of our knowledge, this is the first single speaker corpus in the German language consisting of audio, visual and text modalities of comparable size and temporal extent. We describe the methods used with which we have collected and edited the data which involves downloading the videos, transcripts and other metadata, forced alignment, performing active speaker recognition and face detection to finally curate the single speaker dataset consisting of utterances spoken by Angela Merkel. The proposed pipeline is general and can be used to curate other datasets of similar nature, such as talk show contents. Through various statistical analyses and applications of the dataset in talking face generation and TTS, we show the utility of the dataset. We argue that it is a valuable contribution to the research community, in particular, due to its realistic and challenging material at the boundary between prepared and spontaneous speech.","PeriodicalId":260442,"journal":{"name":"International Conference on Language Resources and Evaluation","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130164688","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Please, Don’t Forget the Difference and the Confidence Interval when Seeking for the State-of-the-Art Status 在寻求最先进的状态时，请不要忘记差异和置信区间

International Conference on Language Resources and Evaluation Pub Date : 2022-05-23 DOI: 10.48550/arXiv.2205.11134

Yves Bestgen

引用次数: 1

TWEET-FID: An Annotated Dataset for Multiple Foodborne Illness Detection Tasks TWEET-FID:用于多种食源性疾病检测任务的注释数据集

International Conference on Language Resources and Evaluation Pub Date : 2022-05-22 DOI: 10.48550/arXiv.2205.10726

Ruofan Hu, Dongyu Zhang, Dandan Tao, Thomas Hartvigsen, Hao Feng, E. Rundensteiner

{"title":"TWEET-FID: An Annotated Dataset for Multiple Foodborne Illness Detection Tasks","authors":"Ruofan Hu, Dongyu Zhang, Dandan Tao, Thomas Hartvigsen, Hao Feng, E. Rundensteiner","doi":"10.48550/arXiv.2205.10726","DOIUrl":"https://doi.org/10.48550/arXiv.2205.10726","url":null,"abstract":"Foodborne illness is a serious but preventable public health problem – with delays in detecting the associated outbreaks resulting in productivity loss, expensive recalls, public safety hazards, and even loss of life. While social media is a promising source for identifying unreported foodborne illnesses, there is a dearth of labeled datasets for developing effective outbreak detection models. To accelerate the development of machine learning-based models for foodborne outbreak detection, we thus present TWEET-FID (TWEET-Foodborne Illness Detection), the first publicly available annotated dataset for multiple foodborne illness incident detection tasks. TWEET-FID collected from Twitter is annotated with three facets: tweet class, entity type, and slot type, with labels produced by experts as well as by crowdsource workers. We introduce several domain tasks leveraging these three facets: text relevance classification (TRC), entity mention detection (EMD), and slot filling (SF). We describe the end-to-end methodology for dataset design, creation, and labeling for supporting model development for these tasks. A comprehensive set of results for these tasks leveraging state-of-the-art single-and multi-task deep learning methods on the TWEET-FID dataset are provided. This dataset opens opportunities for future research in foodborne outbreak detection.","PeriodicalId":260442,"journal":{"name":"International Conference on Language Resources and Evaluation","volume":"85 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127352444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Evaluating Subtitle Segmentation for End-to-end Generation Systems 端到端生成系统的字幕分割评估

International Conference on Language Resources and Evaluation Pub Date : 2022-05-19 DOI: 10.48550/arXiv.2205.09360

Alina Karakanta, François Buet, Mauro Cettolo, Franccois Yvon

{"title":"Evaluating Subtitle Segmentation for End-to-end Generation Systems","authors":"Alina Karakanta, François Buet, Mauro Cettolo, Franccois Yvon","doi":"10.48550/arXiv.2205.09360","DOIUrl":"https://doi.org/10.48550/arXiv.2205.09360","url":null,"abstract":"Subtitles appear on screen as short pieces of text, segmented based on formal constraints (length) and syntactic/semantic criteria. Subtitle segmentation can be evaluated with sequence segmentation metrics against a human reference. However, standard segmentation metrics cannot be applied when systems generate outputs different than the reference, e.g. with end-to-end subtitling systems. In this paper, we study ways to conduct reference-based evaluations of segmentation accuracy irrespective of the textual content. We first conduct a systematic analysis of existing metrics for evaluating subtitle segmentation. We then introduce Sigma, a Subtitle Segmentation Score derived from an approximate upper-bound of BLEU on segmentation boundaries, which allows us to disentangle the effect of good segmentation from text quality. To compare Sigma with existing metrics, we further propose a boundary projection method from imperfect hypotheses to the true reference. Results show that all metrics are able to reward high quality output but for similar outputs system ranking depends on each metric’s sensitivity to error type. Our thorough analyses suggest Sigma is a promising segmentation candidate but its reliability over other segmentation metrics remains to be validated through correlations with human judgements.","PeriodicalId":260442,"journal":{"name":"International Conference on Language Resources and Evaluation","volume":"31 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115829323","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 5

SDS-200: A Swiss German Speech to Standard German Text Corpus SDS-200:瑞士德语语音到标准德语文本语料库

International Conference on Language Resources and Evaluation Pub Date : 2022-05-19 DOI: 10.48550/arXiv.2205.09501

Michel Plüss, Manuela Hurlimann, Marc Cuny, Alla Stockli, Nikolaos Kapotis, Julia Hartmann, Malgorzata Anna Ulasik, C. Scheller, Yanick Schraner, Amit Jain, Jan Deriu, Mark Cieliebak, Manfred Vogel

引用次数: 12

Curras + Baladi: Towards a Levantine Corpus Curras + Baladi:走向黎凡特语料库

International Conference on Language Resources and Evaluation Pub Date : 2022-05-19 DOI: 10.48550/arXiv.2205.09692

Karim El Haff, Mustafa Jarrar, Tymaa Hammouda, F. Zaraket

引用次数: 10

Wojood: Nested Arabic Named Entity Corpus and Recognition using BERT 使用BERT的嵌套阿拉伯语命名实体语料库和识别

International Conference on Language Resources and Evaluation Pub Date : 2022-05-19 DOI: 10.48550/arXiv.2205.09651

Mustafa Jarrar, Mohammed Khalilia, Sana Ghanem

引用次数: 25