{"title":"The LFM-1b Dataset for Music Retrieval and Recommendation","authors":"M. Schedl","doi":"10.1145/2911996.2912004","DOIUrl":"https://doi.org/10.1145/2911996.2912004","url":null,"abstract":"We present the LFM-1b dataset of more than one billion music listening events created by more than 120,000 users of Last.fm. Each listening event is characterized by artist, album, and track name, and further includes a timestamp. On the (anonymous) user level, basic demographics and a selection of more elaborate user descriptors are included. The dataset is foremost intended for benchmarking in music information retrieval and recommendation. To facilitate experimentation in a straightforward manner, it also includes a precomputed user-item-playcount matrix. In addition, sample Python scripts showing how to load the data and perform efficient computations are provided. An implementation of a simple collaborative filtering recommender rounds off the code package. We discuss in detail the LFM-1b dataset's acquisition, availability, statistics, and content, and place it in the context of existing datasets. We also showcase its usage in a simple artist recommendation task, whose results are intended to serve as baseline against which more elaborate techniques can be assessed. The two unique features of the dataset in comparison to existing ones are (i) its substantial size and (ii) a wide range of additional user descriptors that reflect their music taste and consumption behavior.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126151108","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Facial Landmark Detection and Tracking for Facial Behavior Analysis","authors":"Yue Wu","doi":"10.1145/2911996.2912034","DOIUrl":"https://doi.org/10.1145/2911996.2912034","url":null,"abstract":"The face is the most dominant and distinct communication tool of human beings. Automatic analysis of facial behavior allows machines to understand and interpret a human's states and needs for natural interactions. This research focuses on developing advanced computer vision techniques to process and analyze facial images for the recognition of various facial behaviors. Specifically, this research consists of two parts: automatic facial landmark detection and tracking, and facial behavior analysis and recognition using the tracked facial landmark points. In the first part, we develop several facial landmark detection and tracking algorithms on facial images with varying conditions, such as varying facial expressions, head poses and facial occlusions. First, to handle facial expression and head pose variations, we introduce a hierarchical probabilistic face shape model and a discriminative deep face shape model to capture the spatial relationships among facial landmark points under different facial expressions and face poses to improve facial landmark detection. Second, to handle facial occlusion, we improve upon the effective cascade regression framework and propose the robust cascade regression framework for facial landmark detection, which iteratively predicts the landmark visibility probabilities and landmark locations. The second part of this research applies our facial landmark detection and tracking algorithms to facial behavior analysis, including facial action recognition and face pose estimation. For facial action recognition, we introduce a novel regression framework for joint facial landmark detection and facial action recognition. For head pose estimation, we are working on a robust algorithm that can perform head pose estimation under facial occlusion.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"48 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129282889","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Andrea Ferracani, Daniele Pezzatini, M. Bertini, A. Bimbo
{"title":"Item-Based Video Recommendation: An Hybrid Approach considering Human Factors","authors":"Andrea Ferracani, Daniele Pezzatini, M. Bertini, A. Bimbo","doi":"10.1145/2911996.2912066","DOIUrl":"https://doi.org/10.1145/2911996.2912066","url":null,"abstract":"In this paper we propose a method for video recommendation in Social Networks based on crowdsourced and automatic video annotations of salient frames. We show how two human factors, users' self-expression in user profiles and perception of visual saliency in videos, can be exploited in order to stimulate annotations and to obtain an efficient representation of video content features. Results are assessed through experiments conducted on a prototype of social network for video sharing. Several baseline approaches are evaluated and we show how the proposed method improves over them.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129326252","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Baohan Xu, Yanwei Fu, Yu-Gang Jiang, Boyang Albert Li, L. Sigal
{"title":"Video Emotion Recognition with Transferred Deep Feature Encodings","authors":"Baohan Xu, Yanwei Fu, Yu-Gang Jiang, Boyang Albert Li, L. Sigal","doi":"10.1145/2911996.2912006","DOIUrl":"https://doi.org/10.1145/2911996.2912006","url":null,"abstract":"Despite growing research interest, emotion understanding for user-generated videos remains a challenging problem. Major obstacles include the diversity and complexity of video content, as well as the sparsity of expressed emotions. For the first time, we systematically study large-scale video emotion recognition by transferring deep feature encodings. In addition to the traditional, supervised recognition, we study the problem of zero-shot emotion recognition, where emotions in the test set are unseen during training. To cope with this task, we utilize knowledge transferred from auxiliary image and text corpora. A novel auxiliary Image Transfer Encoding (ITE) process is proposed to efficiently encode and generate video representation. We also thoroughly investigate different configurations of convolutional neural networks. Comprehensive experiments on multiple datasets demonstrate the effectiveness of our framework.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116305237","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ilias Gialampoukidis, A. Moumtzidou, T. Tsikrika, S. Vrochidis, Y. Kompatsiaris
{"title":"Retrieval of Multimedia Objects by Fusing Multiple Modalities","authors":"Ilias Gialampoukidis, A. Moumtzidou, T. Tsikrika, S. Vrochidis, Y. Kompatsiaris","doi":"10.1145/2911996.2912068","DOIUrl":"https://doi.org/10.1145/2911996.2912068","url":null,"abstract":"Effective multimedia retrieval requires the combination of the heterogeneous media contained within multimedia objects and the features that can be extracted from them. To this end, we extend a unifying framework that integrates all well-known weighted, graph-based, and diffusion-based fusion techniques that combine two modalities (textual and visual similarities) to model the fusion of multiple modalities. We also provide a theoretical formula for the optimal number of documents that need to be initially selected, so that the memory cost in the case of multiple modalities remains the same as in the case of two modalities. Experiments using two test collections and three modalities (similarities based on visual descriptors, visual concepts, and textual concepts) indicate improvements in the effectiveness over bimodal fusion under the same memory complexity.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126504831","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yiyang Yao, Yingjie Xia, Zhenyu Shan, Zhengguang Liu
{"title":"Learning for Traffic State Estimation on Large Scale of Incomplete Data","authors":"Yiyang Yao, Yingjie Xia, Zhenyu Shan, Zhengguang Liu","doi":"10.1145/2911996.2912037","DOIUrl":"https://doi.org/10.1145/2911996.2912037","url":null,"abstract":"Traffic state estimation is one of the very basic and most important tasks in intelligent transportation systems. Data collected from multiple types of traffic sensors are valuable sources to estimate traffic state. However, those data are always incomplete for the cases such as sensors broken, data missing, etc. The incomplete data can also be used to accurately estimate traffic state by data prediction and data fusion. Firstly, the missing data are predicted based on the data correlations extracted on large amount of historical taxi GPS data. Then, data collected from multiple sensors are fused to get more accurate results. Finally, computing of data fusion is parallelized. The main research focus is placed on learning for extracting the inherent spatio-temporal correlations of traffic data and fusing multimedia traffic data. Accuracy and efficiency experiments are conducted and the experimental results demonstrate that our method can process incomplete data and achieve accurate and real-time traffic state estimation.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128171818","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Serendipity-driven Celebrity Video Hyperlinking","authors":"Shujun Yang, Lei Pang, C. Ngo, B. Huet","doi":"10.1145/2911996.2912029","DOIUrl":"https://doi.org/10.1145/2911996.2912029","url":null,"abstract":"This demo showcases the utility of video hyperlinks with celebrities as the link anchors and their social circles as targets, aiming to help users quickly explore the aboutness of a celebrity by link traversal. Through content analysis, our system embeds hyperlinks into videos such that users can click-and-jump between celebrity faces in different videos to get-to-know their social circles. One peculiar feature is the ability of the system in providing links that maximize users' chance encounter, or serendipitous experience, beyond information need. Our system is enabled by two key components, name-face association and diversity-based ranking, for the aboutness and serendipity features respectively in hyperlinking. The former component assigns names video faces while mining celebrity social networks. The latter performs topic modeling so as to rank videos based on topic diversity.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131393255","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hengliang Zhu, Bin Sheng, Xiao Lin, Yangyang Hao, Lizhuang Ma
{"title":"Foreground Object Sensing for Saliency Detection","authors":"Hengliang Zhu, Bin Sheng, Xiao Lin, Yangyang Hao, Lizhuang Ma","doi":"10.1145/2911996.2912008","DOIUrl":"https://doi.org/10.1145/2911996.2912008","url":null,"abstract":"Many state-of-the-art saliency detection algorithms rely on the boundary prior, but these algorithms simply suppose the boundaries around an image as background regions. Here we propose a fast and effective algorithm for salient object detection. First, a novel method is proposed to approximately locate the foreground object by using the convex hull from Harris corner. On this basis, we divide the saliency values of different regions into two parts and generate the corresponding cue maps (foreground and background), which are combined into a convex hull prior map. Then a new prior based on distance to the convex hull center is proposed to replace the center prior. Finally, the convex hull prior map and the convex hull center-biased map are combined to be the saliency map, which is then optimized to get the final result. Compared with eighteen existing algorithms and tested on several datasets, the present algorithm performs well in terms of precision and recall.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"96 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134025953","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Accurate Aggregation of Local Features by using K-sparse Autoencoder for 3D Model Retrieval","authors":"T. Furuya, Ryutarou Ohbuchi","doi":"10.1145/2911996.2912054","DOIUrl":"https://doi.org/10.1145/2911996.2912054","url":null,"abstract":"Aggregating a set of local features has been used widely to realize recognition or retrieval of multimedia data including 2D images and 3D models. A number of feature aggregation algorithms (e.g., Bag-of-Features, Locality-constrained Linear coding, or Fisher Vector coding) have been proposed. They first learn a codebook, or a set of codewords, by clustering the local features and then encode these local features by using the learned codebook. Despite the great success of these feature aggregation algorithms, we argue that they are not necessarily optimal in terms of accuracy since their codebook learning and feature encoding are computed separately. In this paper, we propose two novel feature aggregation algorithms based on k-Sparse Autoencoder (kSA) that realize more accurate local feature aggregation. Our proposed algorithms, called Database-adaptive kSA (DkSA) aggregation and Per-data-adaptive kSA (PkSA) aggregation, jointly optimize codebook learning and feature encoding. In addition, the kSA-based feature encoding enhances saliency of local features due to k-sparseness constraints and non-negativity constraints. Of the two proposed algorithms, the PkSA aggregation exploits reconstruction error of a local feature derived from the kSA for more accurate aggregated feature. Experimental evaluation using a shape-based 3D model retrieval scenario showed that the retrieval accuracy of our proposed algorithms are superior to the existing feature aggregation algorithms we have compared against.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"85 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133957759","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Diverse Yet Efficient Retrieval using Locality Sensitive Hashing","authors":"V. Rao, Prateek Jain, C. V. Jawahar","doi":"10.1145/2911996.2911998","DOIUrl":"https://doi.org/10.1145/2911996.2911998","url":null,"abstract":"Typical retrieval systems have three requirements: a) Accurate retrieval, i.e., the method should have high precision, b) Diverse retrieval, i.e., the obtained set of samples should be diverse, and c) Retrieval time should be small. However, most of the existing methods address only one or two of the above mentioned requirements. In this work, we present a method based on randomized locality sensitive hashing which tries to address all of the above requirements simultaneously. While earlier hashing-based approaches considered approximate retrieval to be acceptable only for the sake of efficiency, we argue that one can further exploit approximate retrieval to provide impressive trade-offs between accuracy and diversity. We also extend our method to the problem of multi-label prediction, where the goal is to output a diverse and accurate set of labels for a given document in real-time. Finally, we present empirical results on image and text retrieval tasks and show that our method retrieves diverse and accurate images/labels while ensuring 100x-speed-up over the existing diverse retrieval approaches.","PeriodicalId":201992,"journal":{"name":"Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2016-06-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129400660","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}