Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval最新文献

筛选
英文 中文
Multimodal Deep Convolutional Neural Network for Audio-Visual Emotion Recognition 用于视听情感识别的多模态深度卷积神经网络
Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval Pub Date : 2016-06-06 DOI: 10.1145/2911996.2912051
Shiqing Zhang, Shiliang Zhang, Tiejun Huang, Wen Gao
{"title":"Multimodal Deep Convolutional Neural Network for Audio-Visual Emotion Recognition","authors":"Shiqing Zhang, Shiliang Zhang, Tiejun Huang, Wen Gao","doi":"10.1145/2911996.2912051","DOIUrl":"https://doi.org/10.1145/2911996.2912051","url":null,"abstract":"Emotion recognition is a challenging task because of the emotional gap between subjective emotion and the low-level audio-visual features. Inspired by the recent success of deep learning in bridging the semantic gap, this paper proposes to bridge the emotional gap based on a multimodal Deep Convolution Neural Network (DCNN), which fuses the audio and visual cues in a deep model. This multimodal DCNN is trained with two stages. First, two DCNN models pre-trained on large-scale image data are fine-tuned to perform audio and visual emotion recognition tasks respectively on the corresponding labeled speech and face data. Second, the outputs of these two DCNNs are integrated in a fusion network constructed by a number of fully-connected layers. The fusion network is trained to obtain a joint audio-visual feature representation for emotion recognition. Experimental results on the RML audio-visual database demonstrates the promising performance of the proposed method. To the best of our knowledge, this is an early work fusing audio and visual cues in DCNN for emotion recognition. Its success guarantees further research in this direction.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"18 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133890745","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 76
Watching What and How Politicians Discuss Various Topics: A Large-Scale Video Analytics UI 观看政治家讨论各种话题的内容和方式:一个大规模的视频分析UI
Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval Pub Date : 2016-06-06 DOI: 10.1145/2911996.2912025
Emily Song, Joseph G. Ellis, Hongzhi Li, Shih-Fu Chang
{"title":"Watching What and How Politicians Discuss Various Topics: A Large-Scale Video Analytics UI","authors":"Emily Song, Joseph G. Ellis, Hongzhi Li, Shih-Fu Chang","doi":"10.1145/2911996.2912025","DOIUrl":"https://doi.org/10.1145/2911996.2912025","url":null,"abstract":"Accurately gauging the political atmosphere is especially difficult in this day and age, as individuals have access to a constantly growing collection of written and audiovisual news sources. This is especially true with regards to the U.S. presidential election, as there are numerous candidates, countless stories, and opinion articles discussing the merits of each particular candidate. It is therefore challenging for people to make an accurate assessment of what each candidate represents and how they would act if they were elected into office. To address this problem, we present a large-scale dataset comprised of videos of politicians speaking organized by the topics they are speaking about, and a user interface for exploring this interesting dataset. Our interface links people and events to relevant pieces of audiovisual media, and presents the desired information in a meaningful and intuitive manner. Our approach is unique by direct linking to actual speaking by politicians about specific topics, rather than links to textual quotes only. We describe the larger underlying infrastructure, a novel automated system that crawls thousands of internet news sources and 100 television news channels daily, and automatically discovers entities and indexes the content into events and topics. We examine how our user interface provides helpful and unique insights to its users, and give an example of the type of large scale trend analysis that can be performed with our system. Our online demo can be accessed at: http://www.ee.columbia.edu/dvmm/PoliticialSpeakerDemo","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130200787","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Multimodal Visual Pattern Mining with Convolutional Neural Networks 基于卷积神经网络的多模态视觉模式挖掘
Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval Pub Date : 2016-06-06 DOI: 10.1145/2911996.2912033
Hongzhi Li
{"title":"Multimodal Visual Pattern Mining with Convolutional Neural Networks","authors":"Hongzhi Li","doi":"10.1145/2911996.2912033","DOIUrl":"https://doi.org/10.1145/2911996.2912033","url":null,"abstract":"In this paper we describe a novel framework and algorithms for discovering image patch patterns from a large corpus of weakly supervised image-caption pairs generated from news events. Current pattern mining techniques attempt to find patterns that are representative and discriminative, we stipulate that our discovered patterns must also be recognizable by humans and preferably with meaningful names. We propose a new multimodal pattern mining approach that leverages the descriptive captions often accompanying news images to learn semantically meaningful image patch patterns. The mutltimodal patterns are then named using words mined from the associated image captions for each pattern. Our methods also discover named patterns beyond those covered by the existing image datasets like ImageNet. To the best of our knowledge this is the first algorithm developed to automatically mine image patch patterns that have strong semantic meaning specific to high-level news events, and then evaluate these patterns based on that criteria.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129661407","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
SentiCart: Cartography and Geo-contextualization for Multilingual Visual Sentiment SentiCart:多语言视觉情感的制图和地理语境化
Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval Pub Date : 2016-06-06 DOI: 10.1145/2911996.2912022
Brendan Jou, Margaret Yuying Qian, Shih-Fu Chang
{"title":"SentiCart: Cartography and Geo-contextualization for Multilingual Visual Sentiment","authors":"Brendan Jou, Margaret Yuying Qian, Shih-Fu Chang","doi":"10.1145/2911996.2912022","DOIUrl":"https://doi.org/10.1145/2911996.2912022","url":null,"abstract":"Where in the world are pictures of cute animals or ancient architecture most shared from? And are they equally sentimentally perceived across different languages? We demonstrate a series of visualization tools, that we collectively call SentiCart, for answering such questions and navigating the landscape of how sentiment-biased images are shared around the world in multiple languages. We present visualizations using a large-scale, self-gathered geodata corpus of >1.54M geo-references coming from over 235 countries mined from >15K visual concepts over 12 languages. We also highlight several compelling data-driven findings about multilingual visual sentiment in geo-social interactions.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"51 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115978870","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 5
Region Trajectories for Video Semantic Concept Detection 视频语义概念检测的区域轨迹
Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval Pub Date : 2016-06-06 DOI: 10.1145/2911996.2912046
Yuancheng Ye, Xuejian Rong, Xiaodong Yang, Yingli Tian
{"title":"Region Trajectories for Video Semantic Concept Detection","authors":"Yuancheng Ye, Xuejian Rong, Xiaodong Yang, Yingli Tian","doi":"10.1145/2911996.2912046","DOIUrl":"https://doi.org/10.1145/2911996.2912046","url":null,"abstract":"Recently, with the advent of the convolutional neural network (CNN), many CNN-based object detection algorithms have been proposed and achieved encouraging results. In this paper, we introduce an algorithm based on region trajectories to establish the connections between object localizations in individual frames and video sequences. To detect object regions in the individual frames of a video, we enhance the region-based convolutional neural network (R-CNN), by incorporating EdgeBox with the Selective Search to generate candidate region proposals and combining the GoogLeNet with the AlexNet to improve the discriminability of the feature representations. The DeepMatching algorithm is employed in our proposed region trajectory method to track the points in the detected object regions. The experiments are conducted on the validation split of the TRECVID 2015 Localization dataset. As demonstrated by the experimental results, our proposed approach improves the object detection accuracy in both temporal and spatial measurements.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129712128","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 3
On the "Face of Things" 论“事物的表面”
Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval Pub Date : 2016-06-06 DOI: 10.1145/2911996.2930062
Ranran Feng, B. Prabhakaran
{"title":"On the \"Face of Things\"","authors":"Ranran Feng, B. Prabhakaran","doi":"10.1145/2911996.2930062","DOIUrl":"https://doi.org/10.1145/2911996.2930062","url":null,"abstract":"Face is crucial for human identity, while face identification has become crucial to information security. It is important to understand and work with the problems and challenges for all different aspects of facial feature extraction and face identification. In this tutorial, we identify and discuss four research challenges in current Face Detection/Recognition research and related research areas: (1) Unavoidable Facial Feature Alterations, (2) Voluntary Facial Feature Alterations, (3) Uncontrolled Environments, and (4) Accuracy Control on Large-scale Dataset. We also direct several different applications (spin-offs) of facial feature studies in the tutorial.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"57 3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126946000","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 6
Semi-supervised Identification of Rarely Appearing Persons in Video by Correcting Weak Labels 基于弱标签校正的视频中罕见人物的半监督识别
Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval Pub Date : 2016-06-06 DOI: 10.1145/2911996.2912073
Eric Müller, Christian Otto, R. Ewerth
{"title":"Semi-supervised Identification of Rarely Appearing Persons in Video by Correcting Weak Labels","authors":"Eric Müller, Christian Otto, R. Ewerth","doi":"10.1145/2911996.2912073","DOIUrl":"https://doi.org/10.1145/2911996.2912073","url":null,"abstract":"Some recent approaches for character identification in movies and TV broadcasts are realized in a semi-supervised manner by assigning transcripts and/or subtitles to the speakers. However, the labels obtained in this way achieve only an accuracy of $80% - 90%$ and the number of training examples for the different actors is unevenly distributed. In this paper, we propose a novel approach for person identification in video by correcting and extending the training data with reliable predictions to reduce the number of annotation errors. Furthermore, the intra-class diversity of rarely speaking characters is enhanced. To address the imbalance of training data per person, we suggest two complementary prediction scores. These scores are also used to recognize whether or not a face track belongs to a (supporting) character whose identity does not appear in the transcript etc. Experimental results demonstrate the feasibility of the proposed approach, outperforming the current state of the art.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130888087","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 2
Xplore-M-Ego: Contextual Media Retrieval Using Natural Language Queries Xplore-M-Ego:使用自然语言查询的上下文媒体检索
Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval Pub Date : 2016-06-06 DOI: 10.1145/2911996.2912044
Sreyasi Nag Chowdhury, Mateusz Malinowski, A. Bulling, Mario Fritz
{"title":"Xplore-M-Ego: Contextual Media Retrieval Using Natural Language Queries","authors":"Sreyasi Nag Chowdhury, Mateusz Malinowski, A. Bulling, Mario Fritz","doi":"10.1145/2911996.2912044","DOIUrl":"https://doi.org/10.1145/2911996.2912044","url":null,"abstract":"The widespread integration of cameras in hand-held and head-worn devices and the ability to share content online enables a large and diverse visual capture of the world that millions of users build up collectively every day. We envision these images as well as associated meta information, such as GPS coordinates and timestamps, to form a collective visual memory that can be queried while automatically taking the ever-changing context of mobile users into account. As a first step towards this vision, in this work we present Xplore-M-Ego: a novel media retrieval system that allows users to query a dynamic database of images using spatio-temporal natural language queries. We evaluate our system using a new dataset of real image queries as well as through a usability study. One key finding is that there is a considerable amount of inter-user variability in the resolution of spatial relations in natural language utterances. We show that our system can cope with this variability using personalisation through an online learning-based retrieval formulation.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132669446","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 4
Incremental Learning for Fine-Grained Image Recognition 细粒度图像识别的增量学习
Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval Pub Date : 2016-06-06 DOI: 10.1145/2911996.2912069
Liangliang Cao, Jenhao Hsiao, Paloma de Juan, Yuncheng Li, B. Thomee
{"title":"Incremental Learning for Fine-Grained Image Recognition","authors":"Liangliang Cao, Jenhao Hsiao, Paloma de Juan, Yuncheng Li, B. Thomee","doi":"10.1145/2911996.2912069","DOIUrl":"https://doi.org/10.1145/2911996.2912069","url":null,"abstract":"This paper considers the problem of fine-grained image recognition with a growing vocabulary. Since in many real world applications we often have to add a new object category or visual concept with just a few images to learn from, it is crucial to develop a method that is able to generalize the recognition model from existing classes to new classes. Deep convolutional neural networks are capable of constructing powerful image representations; however, these networks usually rely on a logistic loss function that cannot handle the incremental learning problem. In this paper, we present a new method that can efficiently learn a new class given only a limited number of training examples, which we evaluate on the problems of food and clothing recognition. To illustrate the performance of our proposed method on the task of recognizing different kinds of food, when using only 1.3% of training examples per category we achieved about 73% of the performance (as measured by F1-score) compared to when using all available training data.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114479201","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 7
CNN-based Style Vector for Style Image Retrieval 基于cnn的风格向量图像检索
Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval Pub Date : 2016-06-06 DOI: 10.1145/2911996.2912057
S. Matsuo, Keiji Yanai
{"title":"CNN-based Style Vector for Style Image Retrieval","authors":"S. Matsuo, Keiji Yanai","doi":"10.1145/2911996.2912057","DOIUrl":"https://doi.org/10.1145/2911996.2912057","url":null,"abstract":"In this paper, we have examined the effectiveness of \"style matrix\" which is used in the works on style transfer and texture synthesis by Gatys et al. in the context of image retrieval as image features. A style matrix is presented by Gram matrix of the feature maps in a deep convolutional neural network. We proposed a style vector which are generated from a style matrix with PCA dimension reduction. In the experiments, we evaluate image retrieval performance using artistic images downloaded from Wikiarts.org regarding both artistic styles ans artists. We have obtained 40.64% and 70.40% average precision for style search and artist search, respectively, both of which outperformed the results by common CNN features. In addition, we found PCA-compression boosted the performance.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"71 3-4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123588958","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 34
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书