Vincenzo Emanuele Carusotto, G. Pilato, Fabio Persia, Mouzhi Ge
{"title":"User Profiling for Tourist Trip Recommendations using Social Sensing","authors":"Vincenzo Emanuele Carusotto, G. Pilato, Fabio Persia, Mouzhi Ge","doi":"10.1109/ism52913.2021.00036","DOIUrl":"https://doi.org/10.1109/ism52913.2021.00036","url":null,"abstract":"For Point of Interest (POI) recommendations or touristic orienteering applications, it is essential to explore the user preference or potential interests to predict potential POIs or touristic routes. One of the state-of-the-art approaches for inferring the user’s interests is social sensing, which is based on implicit feedback and semantic similarities to extract the user preference. In this paper, we profile the user preferences by social sensing to exploit the similarity between users’ reviews and POI descriptions. The experiment is based on the \"Yelp!\" dataset and provides the labels associated with different businesses considered in the \"Yelp!\" Social Network. The preliminary results show that the proposed model can automatically estimate the user interests and improve the effectiveness of the user profiling procedure. The results also indicate that our proposed model can offer a foundational approach for ranking POIs and touristic routes.","PeriodicalId":440941,"journal":{"name":"2021 IEEE International Symposium on Multimedia (ISM)","volume":"83 4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126076132","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Knowledge-Based Wavelet Filters Prominently Detect Spoofed Speech","authors":"A. M. G. Almeida, R. Guido","doi":"10.1109/ism52913.2021.00014","DOIUrl":"https://doi.org/10.1109/ism52913.2021.00014","url":null,"abstract":"Although voice biometrics has been at the forefront of speech technologies, spoofing attacks have been one of the main issues responsible for avoiding its practical usage in commercial applications. Consequently, this article presents our proposed approach for building Knowledge-based wavelet filters, particularly dedicated to distinguish between genuine and spoofed speech. The main contribution of our strategy, particularly dedicated to identify replay attacks, is the acquisition of knowledge through the so called knowledge coefficients. Our results, obtained upon performing a set of experiments, indicate that the designed wavelet filters successfully classified hundreds of speech signals from our dataset, stimulating further research in this direction.","PeriodicalId":440941,"journal":{"name":"2021 IEEE International Symposium on Multimedia (ISM)","volume":"48 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114844424","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Vötter, Maximilian Mayerl, Günther Specht, Eva Zangerle
{"title":"Novel Datasets for Evaluating Song Popularity Prediction Tasks","authors":"M. Vötter, Maximilian Mayerl, Günther Specht, Eva Zangerle","doi":"10.1109/ism52913.2021.00034","DOIUrl":"https://doi.org/10.1109/ism52913.2021.00034","url":null,"abstract":"Estimating the success of a song before its release is an important music industry task. Current work uses audio descriptors to predict the success (popularity) of a song, where typical measures of success are chart measures such as peak position and streaming measures such as listener-count. Currently, a wide range of datasets is used for that purpose, but most of them are not publicly available; likewise, available datasets are restricted either in size, available features, or popularity measures. This substantially impedes the evaluation of the predictive power of a wide range of models. Therefore, we present two novel datasets called HSP-S and HSP-L based on data from AcousticBrainz, Billboard Hot 100, the Million Song Dataset, and last.fm. Both datasets contain audio features, Mel-spectrograms as well as streaming listener- and play-counts. The larger HSP-L dataset contains 73,482 songs, whereas the smaller HSP-S dataset contains 7,736 songs and additionally features Billboard Hot 100 chart measures. In contrast to previous publicly available datasets, our datasets contain substantially more songs and richer and more diverse features. We solely utilize data from the public domain, allowing to evaluate and compare a wide range of models on our datasets. To demonstrate the use of the datasets, we perform a regression and a classification (popular/unpopular) task on both datasets using a wide variety of models to predict song popularity for all target measures.","PeriodicalId":440941,"journal":{"name":"2021 IEEE International Symposium on Multimedia (ISM)","volume":"61 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130055141","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A sinusoidal signal reconstruction method for the inversion of the mel-spectrogram","authors":"Anastasia Natsiou, Seán O'Leary","doi":"10.1109/ism52913.2021.00048","DOIUrl":"https://doi.org/10.1109/ism52913.2021.00048","url":null,"abstract":"The synthesis of sound via deep learning methods has recently received much attention. Some problems for deep learning approaches to sound synthesis relate to the amount of data needed to specify an audio signal and the necessity of preserving both the long and short time coherence of the synthesised signal. Visual time-frequency representations such as the log-mel-spectrogram have gained in popularity. The log-mel-spectrogram is a perceptually informed representation of audio that greatly compresses the amount of information required for the description of the sound. However, because of this compression, this representation is not directly invertible. Both signal processing and machine learning techniques have previously been applied to the inversion of the log-mel-spectrogram but they both caused audible distortions in the synthesised sounds due to issues of temporal and spectral coherence. In this paper, we outline the application of a sinusoidal model to the ‘inversion’ of the log-mel-spectrogram for pitched musical instrument sounds outperforming state-of-the-art deep learning methods. The approach could be later used as a general decoding step from spectral to time intervals in neural applications.","PeriodicalId":440941,"journal":{"name":"2021 IEEE International Symposium on Multimedia (ISM)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130246052","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Evlampios Apostolidis, Georgios Balaouras, V. Mezaris, I. Patras
{"title":"Combining Global and Local Attention with Positional Encoding for Video Summarization","authors":"Evlampios Apostolidis, Georgios Balaouras, V. Mezaris, I. Patras","doi":"10.1109/ism52913.2021.00045","DOIUrl":"https://doi.org/10.1109/ism52913.2021.00045","url":null,"abstract":"This paper presents a new method for supervised video summarization. To overcome drawbacks of existing RNN-based summarization architectures, that relate to the modeling of long-range frames’ dependencies and the ability to parallelize the training process, the developed model re-lies on the use of self-attention mechanisms to estimate the importance of video frames. Contrary to previous attention-based summarization approaches that model the frames’ dependencies by observing the entire frame sequence, our method combines global and local multi-head attention mechanisms to discover different modelings of the frames’ dependencies at different levels of granularity. Moreover, the utilized attention mechanisms integrate a component that encodes the temporal position of video frames - this is of major importance when producing a video summary. Experiments on two datasets (SumMe and TVSum) demonstrate the effectiveness of the proposed model compared to existing attention-based methods, and its competitiveness against other state-of-the-art supervised summarization approaches. An ablation study that focuses on our main proposed components, namely the use of global and local multi-head attention mechanisms in collaboration with an absolute positional encoding component, shows their relative contributions to the overall summarization performance.","PeriodicalId":440941,"journal":{"name":"2021 IEEE International Symposium on Multimedia (ISM)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124138863","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Steve Blandino, T. Ropitault, Raied Caromi, Jacob Chakareski, M. Khan, N. Golmie
{"title":"Head Rotation Model for Virtual Reality System Level Simulations","authors":"Steve Blandino, T. Ropitault, Raied Caromi, Jacob Chakareski, M. Khan, N. Golmie","doi":"10.1109/ism52913.2021.00016","DOIUrl":"https://doi.org/10.1109/ism52913.2021.00016","url":null,"abstract":"Virtual Reality (VR) promises immersive experiences in diverse areas such as gaming, entertainment, education, healthcare, and remote monitoring. In VR environments, users can navigate 360-degree content by moving or looking around in all directions, by rotating their heads, as in real life. A rapid head rotation can corrupt the wireless link, degrading the user experience. Due to the lack of proper head rotation models, testbeds are usually required to analyze VR systems. In this paper, we propose an open source code package that generates realistic head rotation traces. The code package is based on a simple, yet flexible, time-correlated mathematical model, which is extrapolated from a publicly available VR head rotation measurement-based dataset. We show that the probability density function of head rotation pitch and roll angles can be modeled as Gaussian distributions, while the probability density function of yaw angles can be modeled as a Gaussian mixture distribution. To introduce temporal correlation, we extrapolate the power spectral density of the angular processes, which are modeled with a bi-exponential decay. Finally, we show how the model can support and accelerate the design of future VR systems by proposing the analysis of a distributed Multiple Input Multiple Output (MIMO) system and the design of a situational awareness Machine Learning (ML) based beamforming training for millimeter wave networks.","PeriodicalId":440941,"journal":{"name":"2021 IEEE International Symposium on Multimedia (ISM)","volume":"84 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117175348","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Web Service for Video Smart-Cropping","authors":"Konstantinos Apostolidis, V. Mezaris","doi":"10.1109/ism52913.2021.00011","DOIUrl":"https://doi.org/10.1109/ism52913.2021.00011","url":null,"abstract":"This paper presents a Web service that supports the automatic transformation of a video's aspect ratio. We employ a modified smart-cropping technique from the literature that aims to minimize the loss of semantically important visual content. We integrate this method in an easy-to-use publicly- accessible Web service where a video can be uploaded and automatically transformed to the desired aspect ratio. We also demonstrate that the algorithmic modifications we introduced in the process of building our Web service offer performance gains, when compared to the original method of the literature.","PeriodicalId":440941,"journal":{"name":"2021 IEEE International Symposium on Multimedia (ISM)","volume":"68 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123104807","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Georg Wimmer, R. Schraml, Lukas Lamminger, A. Petutschnigg, A. Uhl
{"title":"Cross-Modality Wood Log Tracing","authors":"Georg Wimmer, R. Schraml, Lukas Lamminger, A. Petutschnigg, A. Uhl","doi":"10.1109/ism52913.2021.00038","DOIUrl":"https://doi.org/10.1109/ism52913.2021.00038","url":null,"abstract":"The proof of origin of logs is becoming increasingly important. In the context of Industry 4.0 and to prevent illegal logging there is an increased interest to track each individual log. In the near future more and more sawmills will be equipped with a computed tomography (CT) scanner. In order to establish wood log traceability from the forest to the sawmill this work investigates log recognition based on RGB log end images captured in the forest and CT log images captured in the sawmill. The advantage of that approach is that CT scanners are already applied in big saw mills to optimize the saw cut and so the logs only have to be recorded once more in the forest which saves time and cost. To bridge the domain shift between CT and RGB images, we apply widely known domain adaption approaches and present a novel filtering approach. Log recognition is done using a convolutional neural network (CNN) based method using the triplet loss for CNN training and a novel shape descriptor. The results (equal error rate of 13%) show that the recognition of logs using different imaging modalities (RGB and CT) is indeed feasible, despite the challenging experimental setup.","PeriodicalId":440941,"journal":{"name":"2021 IEEE International Symposium on Multimedia (ISM)","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124962470","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yusuke Kondo, Xueting Wang, Hiroyuki Seshime, T. Yamasaki
{"title":"Sequential Banner Design Optimization with Deep Reinforcement Learning","authors":"Yusuke Kondo, Xueting Wang, Hiroyuki Seshime, T. Yamasaki","doi":"10.1109/ism52913.2021.00050","DOIUrl":"https://doi.org/10.1109/ism52913.2021.00050","url":null,"abstract":"Many banner images in web advertising are composed of multiple image elements. In this study, we propose a method to optimize the placement of each image element sequentially by self-supervised reinforcement learning. The environment randomly disturbs the position of image elements at first; then, the agent learns how to correct it in a self-supervision manner, enabling the agent to learn how to place each image element for good banner design. For smooth training, we designed new rewards based on the absolute and relative position of image elements. In addition, by incorporating the output of the click-through rate (CTR) prediction model into the reward, the optimized banner design can be expected to have a higher CTR.","PeriodicalId":440941,"journal":{"name":"2021 IEEE International Symposium on Multimedia (ISM)","volume":"10 6","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114034537","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Ahonen, R. G. Youvalari, N. Le, Honglei Zhang, Francesco Cricri, H. R. Tavakoli, M. Hannuksela, Esa Rahtu
{"title":"Learned Enhancement Filters for Image Coding for Machines","authors":"J. Ahonen, R. G. Youvalari, N. Le, Honglei Zhang, Francesco Cricri, H. R. Tavakoli, M. Hannuksela, Esa Rahtu","doi":"10.1109/ism52913.2021.00046","DOIUrl":"https://doi.org/10.1109/ism52913.2021.00046","url":null,"abstract":"Machine-To-Machine (M2M) communication applications and use cases, such as object detection and instance segmentation, are becoming mainstream nowadays. As a consequence, majority of multimedia content is likely to be consumed by machines in the coming years. This opens up new challenges on efficient compression of this type of data. Two main directions are being explored in the literature, one being based on existing traditional codecs, such as the Versatile Video Coding (VVC) standard, that are optimized for human-targeted use cases, and another based on end-to-end trained neural networks. However, traditional codecs have significant benefits in terms of interoperability, real-time decoding, and availability of hardware implementations over end-to-end learned codecs. Therefore, in this paper, we propose learned post-processing filters that are targeted for enhancing the performance of machine vision tasks for images reconstructed by the VVC codec. The proposed enhancement filters provide significant improvements on the target tasks compared to VVC coded images. The conducted experiments show that the proposed post-processing filters provide about 45% and 49% Bjøntegaard Delta Rate gains over VVC in instance segmentation and object detection tasks, respectively.","PeriodicalId":440941,"journal":{"name":"2021 IEEE International Symposium on Multimedia (ISM)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133472466","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}