发布求助
文献互助 智能选刊 最新文献

Proceedings of the Third Workshop on Multimodal Artificial Intelligence最新文献

筛选
英文 中文
On Randomized Classification Layers and Their Implications in Natural Language Generation 随机分类层及其在自然语言生成中的意义
Proceedings of the Third Workshop on Multimodal Artificial Intelligence Pub Date : 2021-06-01 DOI: 10.18653/V1/2021.MAIWORKSHOP-1.2
Gal-Lev Shalev, Gabi Shalev, Joseph Keshet
{"title":"On Randomized Classification Layers and Their Implications in Natural Language Generation","authors":"Gal-Lev Shalev, Gabi Shalev, Joseph Keshet","doi":"10.18653/V1/2021.MAIWORKSHOP-1.2","DOIUrl":"https://doi.org/10.18653/V1/2021.MAIWORKSHOP-1.2","url":null,"abstract":"In natural language generation tasks, a neural language model is used for generating a sequence of words forming a sentence. The topmost weight matrix of the language model, known as the classification layer, can be viewed as a set of vectors, each representing a target word from the target dictionary. The target word vectors, along with the rest of the model parameters, are learned and updated during training. In this paper, we analyze the properties encoded in the target vectors and question the necessity of learning these vectors. We suggest to randomly draw the target vectors and set them as fixed so that no weights updates are being made during training. We show that by excluding the vectors from the optimization, the number of parameters drastically decreases with a marginal effect on the performance. We demonstrate the effectiveness of our method in image-captioning and machine-translation.","PeriodicalId":337590,"journal":{"name":"Proceedings of the Third Workshop on Multimodal Artificial Intelligence","volume":"220 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128616455","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
COIN: Conversational Interactive Networks for Emotion Recognition in Conversation 会话互动网络:会话中情感识别的会话互动网络
Proceedings of the Third Workshop on Multimodal Artificial Intelligence Pub Date : 2021-06-01 DOI: 10.18653/V1/2021.MAIWORKSHOP-1.3
Haidong Zhang, Yekun Chai
{"title":"COIN: Conversational Interactive Networks for Emotion Recognition in Conversation","authors":"Haidong Zhang, Yekun Chai","doi":"10.18653/V1/2021.MAIWORKSHOP-1.3","DOIUrl":"https://doi.org/10.18653/V1/2021.MAIWORKSHOP-1.3","url":null,"abstract":"Emotion recognition in conversation has received considerable attention recently because of its practical industrial applications. Existing methods tend to overlook the immediate mutual interaction between different speakers in the speaker-utterance level, or apply single speaker-agnostic RNN for utterances from different speakers. We propose COIN, a conversational interactive model to mitigate this problem by applying state mutual interaction within history contexts. In addition, we introduce a stacked global interaction module to capture the contextual and inter-dependency representation in a hierarchical manner. To improve the robustness and generalization during training, we generate adversarial examples by applying the minor perturbations on multimodal feature inputs, unveiling the benefits of adversarial examples for emotion detection. The proposed model empirically achieves the current state-of-the-art results on the IEMOCAP benchmark dataset.","PeriodicalId":337590,"journal":{"name":"Proceedings of the Third Workshop on Multimodal Artificial Intelligence","volume":"48 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116438602","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
Semantic Aligned Multi-modal Transformer for Vision-LanguageUnderstanding: A Preliminary Study on Visual QA 面向视觉语言理解的语义对齐多模态转换器:视觉QA的初步研究
Proceedings of the Third Workshop on Multimodal Artificial Intelligence Pub Date : 2021-06-01 DOI: 10.18653/V1/2021.MAIWORKSHOP-1.11
H. Ding, L. Li, Zhiting Hu, Yi Xu, Dilek Z. Hakkani-Tür, Zheng Du, Belinda Zeng
{"title":"Semantic Aligned Multi-modal Transformer for Vision-LanguageUnderstanding: A Preliminary Study on Visual QA","authors":"H. Ding, L. Li, Zhiting Hu, Yi Xu, Dilek Z. Hakkani-Tür, Zheng Du, Belinda Zeng","doi":"10.18653/V1/2021.MAIWORKSHOP-1.11","DOIUrl":"https://doi.org/10.18653/V1/2021.MAIWORKSHOP-1.11","url":null,"abstract":"Recent vision-language understanding approaches adopt a multi-modal transformer pre-training and finetuning paradigm. Prior work learns representations of text tokens and visual features with cross-attention mechanisms and captures the alignment solely based on indirect signals. In this work, we propose to enhance the alignment mechanism by incorporating image scene graph structures as the bridge between the two modalities, and learning with new contrastive objectives. In our preliminary study on the challenging compositional visual question answering task, we show the proposed approach achieves improved results, demonstrating potentials to enhance vision-language understanding.","PeriodicalId":337590,"journal":{"name":"Proceedings of the Third Workshop on Multimodal Artificial Intelligence","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124129553","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Multi Task Learning based Framework for Multimodal Classification 基于多任务学习的多模态分类框架
Proceedings of the Third Workshop on Multimodal Artificial Intelligence Pub Date : 2021-06-01 DOI: 10.18653/V1/2021.MAIWORKSHOP-1.5
Danting Zeng
{"title":"Multi Task Learning based Framework for Multimodal Classification","authors":"Danting Zeng","doi":"10.18653/V1/2021.MAIWORKSHOP-1.5","DOIUrl":"https://doi.org/10.18653/V1/2021.MAIWORKSHOP-1.5","url":null,"abstract":"Large-scale multi-modal classification aim to distinguish between different multi-modal data, and it has drawn dramatically attentions since last decade. In this paper, we propose a multi-task learning-based framework for the multimodal classification task, which consists of two branches: multi-modal autoencoder branch and attention-based multi-modal modeling branch. Multi-modal autoencoder can receive multi-modal features and obtain the interactive information which called multi-modal encoder feature, and use this feature to reconstitute all the input data. Besides, multi-modal encoder feature can be used to enrich the raw dataset, and improve the performance of downstream tasks (such as classification task). As for attention-based multimodal modeling branch, we first employ attention mechanism to make the model focused on important features, then we use the multi-modal encoder feature to enrich the input information, achieve a better performance. We conduct extensive experiments on different dataset, the results demonstrate the effectiveness of proposed framework.","PeriodicalId":337590,"journal":{"name":"Proceedings of the Third Workshop on Multimodal Artificial Intelligence","volume":"45 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114841149","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Validity-Based Sampling and Smoothing Methods for Multiple Reference Image Captioning 基于有效性的多参考图像字幕采样和平滑方法
Proceedings of the Third Workshop on Multimodal Artificial Intelligence Pub Date : 2021-06-01 DOI: 10.18653/V1/2021.MAIWORKSHOP-1.6
Shunta Nagasawa, Yotaro Watanabe, H. Iyatomi
{"title":"Validity-Based Sampling and Smoothing Methods for Multiple Reference Image Captioning","authors":"Shunta Nagasawa, Yotaro Watanabe, H. Iyatomi","doi":"10.18653/V1/2021.MAIWORKSHOP-1.6","DOIUrl":"https://doi.org/10.18653/V1/2021.MAIWORKSHOP-1.6","url":null,"abstract":"In image captioning, multiple captions are often provided as ground truths, since a valid caption is not always uniquely determined. Conventional methods randomly select a single caption and treat it as correct, but there have been few effective training methods that utilize multiple given captions. In this paper, we proposed two training technique for making effective use of multiple reference captions: 1) validity-based caption sampling (VBCS), which prioritizes the use of captions that are estimated to be highly valid during training, and 2) weighted caption smoothing (WCS), which applies smoothing only to the relevant words the reference caption to reflect multiple reference captions simultaneously. Experiments show that our proposed methods improve CIDEr by 2.6 points and BLEU4 by 0.9 points from baseline on the MSCOCO dataset.","PeriodicalId":337590,"journal":{"name":"Proceedings of the Third Workshop on Multimodal Artificial Intelligence","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132113891","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整，为了账户安全，请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们：info@booksci.cn Book学术提供免费学术资源搜索服务，方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号：604180095
Book学术官方微信
小红书