{"title":"A Case Study of Machine Learning Classifiers for Real-Time Adaptive Resolution Prediction in Video Coding","authors":"Madhukar Bhat, Jean-Marc Thiesse, P. Callet","doi":"10.1109/icme46284.2020.9102934","DOIUrl":"https://doi.org/10.1109/icme46284.2020.9102934","url":null,"abstract":"At lower bit-rate encoding video in real-time with a reasonable viewing quality is challenging. Content adaptive per-title encoding is usually leveraged for OTT/VOD delivery by selecting the optimal resolutions and qualities of a given video using multiple encodings. Built on such powerful resolution selection principles, this paper introduces an on the fly resolution prediction without requiring multiple encoding with the help of machine learning which is suitable for real-time video delivery. Two machine learning networks are defined based on the resolution of the previous decision period. Three types of machine learning classifiers: weighted SVM, Random Forests (RF), and custom-designed Multi-Layer Perceptron (MLP) are tested. Suitability of classifiers for real-time resolution prediction is discussed based on the accuracy, BDrate performances, and impact of misclassification on encoding performance and hardware implementability. The proposed solution offers a promising average bit-rate savings upto 12.6%.","PeriodicalId":396992,"journal":{"name":"2020 IEEE International Conference on Multimedia and Expo (ICME)","volume":"14 9","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114100187","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Srnmfrb: A Deep Light-Weight Super Resolution Network Using Multi-Receptive Field Feature Generation Residual Blocks","authors":"Alireza Esmaeilzehi, M. Ahmad, M. Swamy","doi":"10.1109/ICME46284.2020.9102951","DOIUrl":"https://doi.org/10.1109/ICME46284.2020.9102951","url":null,"abstract":"Deep neural networks use a nonlinear end-to-end mapping in order to transform a low resolution image to the high resolution one. Residual blocks facilitate the flow of the information in deep neural networks and enhance the network performance. In this paper, a new residual block that enhances the representational capability of a super resolution network is proposed. The proposed residual block combines the features generated in various receptive fields using different hierarchical levels of convolution operations or convolution operations in conjunction with the space-to-depth and depth-to-space operations in order to provide a rich set of residual features. The experimental results demonstrate the superiority of the super resolution network using the proposed residual block over the state-of-the-art light-weight super resolution networks in terms of objective and subjective metrics.","PeriodicalId":396992,"journal":{"name":"2020 IEEE International Conference on Multimedia and Expo (ICME)","volume":"77 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116356110","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Open Set Semantic Segmentation With Statistical Test And Adaptive Threshold","authors":"Z. Cui, Longshi Wu, Ruixuan Wang","doi":"10.1109/icme46284.2020.9102712","DOIUrl":"https://doi.org/10.1109/icme46284.2020.9102712","url":null,"abstract":"Semantic segmentation in the open world is prerequisite when deploying a well-trained segmentation model in real scenarios, where objects of unseen classes during model training may often appear in future new images to be segmented by the model. However, such open set semantic segmentation task has been rarely explored before. In this study, making use of the large number of pixel-level prediction uncertainties for each image, we proposed applying the non-parametric statistical test to detect whether objects of unseen classes appear in a new image, and an adaptive threshold method to automatically segment each pixel into either one of the known classes or the unknown class. Experiments on the natural image dataset showed that the proposed method significantly outperforms multiple strong baseline methods.","PeriodicalId":396992,"journal":{"name":"2020 IEEE International Conference on Multimedia and Expo (ICME)","volume":"26 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124540246","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Steganographer Detection Via Enhancement-Aware Graph Convolutional Network","authors":"Zhi-Li Zhang, Mingjie Zheng, S. Zhong, Yan Liu","doi":"10.1109/icme46284.2020.9102817","DOIUrl":"https://doi.org/10.1109/icme46284.2020.9102817","url":null,"abstract":"Steganographer detection aims to find guilty users who hide secret information in images or other multimedia data in the social network. In existing work, the distances between users are calculated based on the distributions of all images shared by the corresponding users, then users lying an abnormal distance from others are detected as guilty users. This flattened method is difficult to grasp the nuances of the guilty and innocent users. In this paper, we are the first to propose a graph-based deep learning framework for steganographer detection. The proposed Enhancement-aware Graph Convolutional Network (EGCN) represents each user as a weighted complete graph and learns to highlight the differences between guilty users and innocent users based on the structured graph. Compared with the state-of-the-art method and other representative graph-based models, the proposed framework demonstrates its effectiveness across image domains, and even under the context of large-scale social media scenario.","PeriodicalId":396992,"journal":{"name":"2020 IEEE International Conference on Multimedia and Expo (ICME)","volume":"56 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125604095","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Learning Converse-Level Multimodal Embedding to Assess Social Deficit Severity for Autism Spectrum Disorder","authors":"Chin-Po Chen, S. S. Gau, Chi-Chun Lee","doi":"10.1109/icme46284.2020.9102869","DOIUrl":"https://doi.org/10.1109/icme46284.2020.9102869","url":null,"abstract":"Developing algorithms to automatically assess mental constructs using media data of human behaviors is becoming important, especially relevant for mental health applications. In this work, we focus on a critically prevalent neurodevelopment disorder, i.e., the autism spectrum disorder (ASD). While researchers have worked on automatic differentiation of ASD from healthy control using a variety of behavior modalities, few works have modeled the severity of ASD behavior symptoms in the existing clinical practice. Thus, we propose to learn a converse-level multimodal (speech and text) embedding derived during a severity assessment interview, i.e., the Autism Diagnosis Observation Schedule (ADOS), that considers the intricate interaction behaviors between the investigator and the participant. Further by fusing two attentional GRUs with this multimodal embedding, our approach achieves an averaged regression score of 0.567 on four items of socio-communicative constructs in the ADOS. Our analysis results suggest that the number of words uttered by both the investigator and the participant is a major predictor.","PeriodicalId":396992,"journal":{"name":"2020 IEEE International Conference on Multimedia and Expo (ICME)","volume":"2 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115819969","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Hardware-Aware Transformable Architecture Search with Efficient Search Space","authors":"Yuhang Jiang, Xin Wang, Wenwu Zhu","doi":"10.1109/icme46284.2020.9102721","DOIUrl":"https://doi.org/10.1109/icme46284.2020.9102721","url":null,"abstract":"While Neural Architecture Search (NAS) discovers the optimal topology structure, Transformable Architecture Search (TAS) aims to search for the best width and depth, which is more challenging due to the larger search space. Since FLOPs is inconsistent with the actual latency, hardware-aware TAS uses the inference latency to evaluate the efficiency. However, most existing work focuses on the search strategy, ignoring the critical role of the search space in affecting the actual efficiency. Motivated by it, we study hardware-aware TAS by considering the search space, to the best of our knowledge, for the first time. We propose a hardware-aware transformable architecture search (HTAS) framework to discover the optimal architecture for different hardware. The core of our method is a novel hardware-aware search space, which provides efficient channel choices for the search strategy to sample efficient architectures. Experiments on CIFAR datasets demonstrate the superiority of HTAS over the state-of-the-art method.","PeriodicalId":396992,"journal":{"name":"2020 IEEE International Conference on Multimedia and Expo (ICME)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134110385","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yuri Mikawa, Tomohiro Sueishi, Yoshihiro Watanabe, M. Ishikawa
{"title":"Projection Mapping System To A Widely Dynamic Sphere With Circumferential Markers","authors":"Yuri Mikawa, Tomohiro Sueishi, Yoshihiro Watanabe, M. Ishikawa","doi":"10.1109/icme46284.2020.9102813","DOIUrl":"https://doi.org/10.1109/icme46284.2020.9102813","url":null,"abstract":"Image projection on spheres and their surroundings have been researched for all-around display and motion visualization. However, wide-range projection onto a dynamic sphere can suffer from tracking errors and projection latency. We propose a high-speed projection mapping system for dynamic spheres, and circumferential markers for sphere posture estimation. Marker detection based on ellipse appearance can easily start tracking, and is robust against interactive occlusion; therefore, the system enables projection mapping and rotational visualization for a dynamically moving sphere. We experimentally confirmed sufficient pose estimation accuracy of the circumferential markers compared with initialized dot markers, and sufficient tracking ability against initialization, occlusion, and depth-direction movement. Demonstrations showed accurate, rotation-visualized projection onto a dynamic sphere, which is useful for sports practice applications.","PeriodicalId":396992,"journal":{"name":"2020 IEEE International Conference on Multimedia and Expo (ICME)","volume":"30 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134324292","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xiangyu Zhu, Xiaoguang Han, Wei Zhang, Jian Zhao, Ligang Liu
{"title":"Learning Intrinsic Decomposition of Complex-Textured Fashion Images","authors":"Xiangyu Zhu, Xiaoguang Han, Wei Zhang, Jian Zhao, Ligang Liu","doi":"10.1109/ICME46284.2020.9102901","DOIUrl":"https://doi.org/10.1109/ICME46284.2020.9102901","url":null,"abstract":"In this paper, we present a deep learning approach for shading decomposition from single garment images with complex textured albedos. We propose a multi-stream generator to infer albedo, shading, normal and lighting in a unified network and train it in a novel local-global adversarial learning framework. To support the training, we carefully build a large dataset with synthetic garment photos under various lighting conditions and textured albedos. Both quantitative and qualitative evaluations, on synthetic and real images, demonstrate the superiority of our method against the state-of-the-art.","PeriodicalId":396992,"journal":{"name":"2020 IEEE International Conference on Multimedia and Expo (ICME)","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131848985","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Peng Chen, Jin Liu, Tao Liang, Guangzhi Zhou, Hongchao Gao, Jiao Dai, Jizhong Han
{"title":"FSSPOTTER: Spotting Face-Swapped Video by Spatial and Temporal Clues","authors":"Peng Chen, Jin Liu, Tao Liang, Guangzhi Zhou, Hongchao Gao, Jiao Dai, Jizhong Han","doi":"10.1109/icme46284.2020.9102914","DOIUrl":"https://doi.org/10.1109/icme46284.2020.9102914","url":null,"abstract":"Recent advances in face generation and manipulation have enabled the creation of sophisticated face-swapped videos, also known as DeepFakes, which brings great potential threats to our society. Hence, it is crucial to develop effective approaches to distinguish them. Currently, face-swapped videos produced by existing methods are prone to exhibit some subtle spatial and temporal manipulated traces, which can be utilized as distinctive clues for face-swapped video detection. In this paper, we propose a unified framework, named FSSpotter, to explore rich spatial and temporal information in the video simultaneously. It consists of a Spatial Feature Extractor (SFE), which aims to discover spatial evidences within a single frame, and a Temporal Feature Aggregator (TFA), which is responsible for capturing temporal inconsistencies between frames. Moreover, a novel data processing strategy is adopted to highlight the inconsistencies of forged face with its surrounding regions. The evaluations on Deepfakes of FaceForensics++, DeepfakeTIMIT, UADFV and Celeb-DF datasets demonstrate that the proposed approach achieves better or comparable performance on AUC scores.","PeriodicalId":396992,"journal":{"name":"2020 IEEE International Conference on Multimedia and Expo (ICME)","volume":"48 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131056055","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"2Ser-Vgsr-Net: A Two-Stage Enhancement Reconstruction Based On Video Group Sparse Representation Network For Compressed Video Sensing","authors":"Yunyi Xuan, Chunling Yang","doi":"10.1109/ICME46284.2020.9102849","DOIUrl":"https://doi.org/10.1109/ICME46284.2020.9102849","url":null,"abstract":"Compressed sensing (CS) requires fewer measurements than the Nyquist theory, making it excellent potential for video compression. However, the complex computation and long-running time limit the traditional compressed video sensing (CVS) methods in real-time application. In this paper, we proposed a fast CVS reconstruction based on deep network named 2sER-VGSR-Net. We first perform ISTA-Net+ as initial reconstruction. To exploit temporal and spatial correlation intrinsically, we construct a video inter-frame group by extracting blocks from the current and reference frames while establishing a sparse representation by network, called VGSR-Net. Different from traditional CVS methods, the group proposed in this paper contains fewer blocks thanks to the accurate alignment by STMC-Net. The inter-frame reconstruction comprises two stages, of which the first stage gets primary enhancement for motion compensation, and the latter performs as residual reconstruction to recover the details. Experiments show that the proposed 2sER-VGSR-Net outperforms the existing state-of-art CVS reconstruction algorithms.","PeriodicalId":396992,"journal":{"name":"2020 IEEE International Conference on Multimedia and Expo (ICME)","volume":"48 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130741608","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}