{"title":"Spatiotemporal-Spectral Graph Convolutional Networks For Skeleton-Based Action Recognition","authors":"Shuo Chen, Ke Xu, Xinghao Jiang, Tanfeng Sun","doi":"10.1109/ICMEW53276.2021.9455987","DOIUrl":"https://doi.org/10.1109/ICMEW53276.2021.9455987","url":null,"abstract":"Skeleton-based action recognition has been a hot topic with the increasing development of Graph Convolutional Networks(GCNs). Previous work constructed the skeleton sequences into graphs and focused on extracting spatial-temporal information from various actions. However, they ignored the hidden information in the spectral domain of the whole graphs. In this paper, a novel graph network based on both spatio-temporal information and spectral-domain information is proposed(SS-GCN), adopting a two-stream graph topology and can be trained in an end-to-end manner. Besides, along with other GCN methods that optimize only the spatial-temporal graph, our spectral stream helps in further performance improvements. Our method(SS-GCN) is evaluated on two large skeleton-based datasets, NTU-RGBD and Kinetics-Skeleton. The experiment results demonstrate the effectiveness of SS-GCN.","PeriodicalId":228350,"journal":{"name":"2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)","volume":"64 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"125097412","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jichao Kan, Kun Hu, Zhiyong Wang, Qiuxia Wu, M. Hagenbuchner, A. Tsoi
{"title":"Topic-Guided Local-Global Graph Neural Network For Image Captioning","authors":"Jichao Kan, Kun Hu, Zhiyong Wang, Qiuxia Wu, M. Hagenbuchner, A. Tsoi","doi":"10.1109/ICMEW53276.2021.9455991","DOIUrl":"https://doi.org/10.1109/ICMEW53276.2021.9455991","url":null,"abstract":"Image captioning is to generate textual descriptions for a given image by analyzing its visual semantics. It can be applied for numerous applications such as surveillance, where generating descriptions of images enables a more efficient workflow. However, accurate descriptions require to formulate the interactions among visual objects and semantics, which have not been adequately exploited yet. Therefore, a novel architecture is proposed, namely topic-guided local-global graph neural network, to address the interactions in a two-level scheme. Local information is characterized through visual objects and semantic graphs are introduced to formulate their relations. Global information is characterized with a topic graph to analyze captioning context and guides the semantic graphs for captioning. Particularly, graph convolutions and graph transformers with a connection between the adjacency matrices are explored. Experimental results on MS-COCO dataset demonstrate the effectiveness of our proposed method.","PeriodicalId":228350,"journal":{"name":"2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124046260","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
H. Hu, Chen Wang, Tao Jiang, Zhixian Guo, Yonggang Han
{"title":"Robust and Efficient Facial Landmark Localization","authors":"H. Hu, Chen Wang, Tao Jiang, Zhixian Guo, Yonggang Han","doi":"10.1109/ICMEW53276.2021.9455952","DOIUrl":"https://doi.org/10.1109/ICMEW53276.2021.9455952","url":null,"abstract":"Facial landmark localization is an essential component for facial recognition which is a vital technology for patients tracking in the situation of a global pandemic of COVID-19. Parameter amount and computation cost are two key factors affecting the application of facial landmark localization. In addition, the intrinsic imbalance of head pose in existing datasets has a great impact on the model generalization. To address these problems, this paper proposed a lightweight model LiteFace with multi-knowledge distillation and a pose-aware resampling strategy. Specifically, we proposed to extract multi-scale features and enhance the receptive filed of network by delicate model design to obtain an efficient architecture and improved the performance of the lightweight model by multi-knowledge distillation which combines knowledge distillation and mutual learning in a two-stage manner. To further improve the model performance and boost the generalization, we proposed a pose-aware re-sampling strategy that generates samples with different head poses and utilized the LaPa dataset to generate masked face images to increase data diversity. We conducted extensive ablation studies on the model design, multi-knowledge distillation and resampling strategy. The proposed method achieves 1.93% and 3.04% NME on the JD-landmark-mask val and test dataset respectively. Finally, LiteFace wins the third place in the 3rd Grand Challenge of 106-Point Facial Landmark Localization.","PeriodicalId":228350,"journal":{"name":"2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)","volume":"35 4","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"120920828","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yuyang Sha, Jie Zhang, Xiao Liu, Zhongqin Wu, S. Shan
{"title":"Efficient Face Alignment Network For Masked Face","authors":"Yuyang Sha, Jie Zhang, Xiao Liu, Zhongqin Wu, S. Shan","doi":"10.1109/ICMEW53276.2021.9456012","DOIUrl":"https://doi.org/10.1109/ICMEW53276.2021.9456012","url":null,"abstract":"The sudden outbreak and global spread of the COVID-19 have severely disrupted normal life of everyone. Masks have become an effective way to suppress the transmission of the COVID-19. The widespread use of masks has brought sever challenges to many face-based algorithms, which would severely damage the accuracy of model. As the basis of various facial analysis algorithms, the performance of facial landmark detection could directly affect the precision of them. However, the existing methods cannot effectively locate facial landmarks of masked faces. In order to improve the robustness to masked face, we propose an efficient masked face alignment network named MaskFAN. Specifically, we introduce depthwise separable convolution and group operation to build a lightweight backbone. A novel loss function and data augmentation module are also used to boost the performance. Extensive experiments are conducted on masked face dataset, and the results show that our approach surpasses the state-of-the-art methods by a significant margin with small model size and limited computation cost.","PeriodicalId":228350,"journal":{"name":"2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131442448","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jingwen Yan, Yuting Zhu, Zilin Liang, Yisheng Zhu, Keer Wu, Zhinan Lin
{"title":"Tire Pattern Classification Based On Few-Shot Learning","authors":"Jingwen Yan, Yuting Zhu, Zilin Liang, Yisheng Zhu, Keer Wu, Zhinan Lin","doi":"10.1109/ICMEW53276.2021.9455988","DOIUrl":"https://doi.org/10.1109/ICMEW53276.2021.9455988","url":null,"abstract":"Learning from a limited number of samples is challenging since the learned model can easily become overfitted based on the biased distribution formed by only a few training examples. In this paper, we design a novel few-shot learning method to solve the problem of tire pattern classification proposed by Xi’an University of Posts and Telecommunications laboratory. The proposed method consists of two steps. On the one hand, we calibrate the distribution of these fewsample classes by transferring statistics from the classes with sufficient features (FD). On the other hand, an adequate number of examples can be sampled from the feature distribution to expand the inputs to the classifier (SVM). Experimental results on the Tire pattern dataset demonstrate that, compared with the existing few-shot learning models, the proposed FT with SVM provides noticeably more robust and higher performance, thus making it a useful tool for practical application scenarios.","PeriodicalId":228350,"journal":{"name":"2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)","volume":"101 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123164118","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Wenjie Zou, Shixuan Feng, Xionghui Mao, Fuzheng Yang, Zhibin Ma
{"title":"Enhancing Quality Of Experience For Cloud Virtual Reality Gaming: An Object-Aware Video Encoding","authors":"Wenjie Zou, Shixuan Feng, Xionghui Mao, Fuzheng Yang, Zhibin Ma","doi":"10.1109/ICMEW53276.2021.9455970","DOIUrl":"https://doi.org/10.1109/ICMEW53276.2021.9455970","url":null,"abstract":"Cloud gaming is an ideal solution to provide the desired GPU rendering for the Virtual Reality (VR) gaming while reducing the requirements for the weight, price, and capabilities of the head-mounted display. However, transmitting the high-resolution video requires a large amount of network bandwidth. In this paper, we present a cloud VR gaming system that significantly improves the coding efficiency by using a proposed Object-Aware Video Encoding (OAVE). Different from the video service, the game engine can offer information about the objects in the game scene. By combining this information and the eye-tracking data, the encoder compressed the rendered video in an object-dependent way that sets the quantization parameter according to the object of the user’s gaze. Through the subjective experiment conducted on the prototype of the system, above 40% to 47% bandwidth saving can be achieved compared with the conventional VR gaming system without the proposed optimization.","PeriodicalId":228350,"journal":{"name":"2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)","volume":"175 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114845929","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An Ensemble Learning-Based No Reference Qoe Model For User Generated Contents","authors":"Duc V. Nguyen, Huyen T. T. Tran, T. Thang","doi":"10.1109/ICMEW53276.2021.9455959","DOIUrl":"https://doi.org/10.1109/ICMEW53276.2021.9455959","url":null,"abstract":"The explosive growth of user-generated content (UGC) videos demands new Quality of Experience model for this new type of content. In this paper, we propose a novel No Reference QoE model for UGC videos. The proposed model takes into account not only bitstream-level and packet-level information, but also encoding parameters. Features are extracted from input videos at frame level, then combined by a pooling layer. In addition, ensemble learning is utilized to improve overall prediction performance. Experimental results show that our model can predict MOS values of UGC videos with very high accuracy. Also, the proposed model is simple, and so can be used for real-time QoE monitoring.","PeriodicalId":228350,"journal":{"name":"2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115598409","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Yongxiong Wang, Chuanfei Hu, Guangpeng Wang, Xu Lin
{"title":"Contrastive Representation For Few-Shot Vehicle Footprint Recognition","authors":"Yongxiong Wang, Chuanfei Hu, Guangpeng Wang, Xu Lin","doi":"10.1109/ICMEW53276.2021.9455981","DOIUrl":"https://doi.org/10.1109/ICMEW53276.2021.9455981","url":null,"abstract":"Vehicle footprint recognition provides valuable clues in criminal case and traffic accident management. To further implement vehicle footprint recognition system in the few-shot scenario, we introduce contrastive learning in few-shot learning diagram. Specifically, we first exploit data preprocessing to alleviate the pattern inconsistencies of tire samples. Then, Noise Contrastive Estimation (NCE) is utilized as data augmentation technique in contrastive learning for constructing a few-shot representation model. In order to avoid the randomness of the experiment, the average value of the results in experiments are adopted with 95% confidence. Experimental results demonstrate the effectiveness of our method, which performs a higher accuracy for ICME2021 Grand Challenge: few-shot learning for vehicle footprint recognition.","PeriodicalId":228350,"journal":{"name":"2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)","volume":"52 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128475480","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. Song, Peixi Peng, Yunpeng Zhai, Chong Zhang, Yonghong Tian
{"title":"Adaptive Multi-Scale Semantic Fusion Network For Zero-Shot Learning","authors":"J. Song, Peixi Peng, Yunpeng Zhai, Chong Zhang, Yonghong Tian","doi":"10.1109/ICMEW53276.2021.9455945","DOIUrl":"https://doi.org/10.1109/ICMEW53276.2021.9455945","url":null,"abstract":"Zero-shot learning aims at accurately recognizing unseen objects by learning matrices that bridge the gap between visual information and semantic attributes. Existing approaches predominantly focus on learning the proper mapping function for visual-semantic embedding while neglecting the effect of learning discriminative semantic features, which leads to severe semantic ambiguity. We propose a practical Adaptive Multi-scale Semantic Fusion (AMSF) framework to perform object-based multi-scale attribute attention for semantic disambiguation. Considering both low-level visual information and global class-level features that relate to this ambiguity, the proposed method jointly learns cooperative global and local semantic attributes from different scales. Moreover, with the joint supervision of embedding softmax loss and class-center triplet loss, the model is encouraged to learn high discriminative semantic features and visual features with high interclass dispersion and intra-class compactness. The method is evaluated on CUB, AwA2, and SUN datasets, and the experimental results indicate the method achieves state-of-the-art performance.","PeriodicalId":228350,"journal":{"name":"2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129970668","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zhipeng Wang, Chunping Hou, Yang Liu, Bangbang Ge, Guanghui Yue, Chunying Song
{"title":"Image Quality Assessment Driven Self-Supervised Anomaly Detection","authors":"Zhipeng Wang, Chunping Hou, Yang Liu, Bangbang Ge, Guanghui Yue, Chunying Song","doi":"10.1109/ICMEW53276.2021.9455946","DOIUrl":"https://doi.org/10.1109/ICMEW53276.2021.9455946","url":null,"abstract":"Anomaly detection is a challenging task due to the bottleneck of anomalous sample collection. Recently, self-supervised learning has shown great potential for anomaly detection. However, these methods use task-related attributes as self-supervised signals and are suitable for specific application scenarios. In this paper, we propose a more general self-supervised method to detect anomaly which regards image quality as a supervised signal. Specially, we convert anomaly detection task into an image quality restoration task which contains distortion and restoration processing. Firstly, we introduce content distortion which erases hardness blocks in normal image based on no reference quality assessment method. Meanwhile, we introduce resolution distortion which down-samples normal images to low-resolution images preserving the essential appearance information. Then, a restoration network is trained to repair such distortions that encourages the model to learn detail information and the content of hardness regions in normal image. Extensive experiments on public benchmark datasets verify the effectiveness and generality of the proposed method.","PeriodicalId":228350,"journal":{"name":"2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)","volume":"54 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2021-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130252959","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}