2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)最新文献

筛选
英文 中文
Deep Semantic Hashing with Structure-Semantic Disagreement Correction via Hyperbolic Metric Learning 基于双曲度量学习的结构-语义纠错深度语义哈希
2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP) Pub Date : 2022-09-26 DOI: 10.1109/MMSP55362.2022.9948733
Fazail Amin, Arijit Mondal, Jimson Mathew
{"title":"Deep Semantic Hashing with Structure-Semantic Disagreement Correction via Hyperbolic Metric Learning","authors":"Fazail Amin, Arijit Mondal, Jimson Mathew","doi":"10.1109/MMSP55362.2022.9948733","DOIUrl":"https://doi.org/10.1109/MMSP55362.2022.9948733","url":null,"abstract":"Semantic hashing is a crucial component of content based search and retrieval systems. To achieve an effective semantic hashing for images, it is essential to map them to hash space in a way that preserves the semantic information. Most state-of-the-art deep semantic hashing approaches do not fully take into account the structural information and the inherent hierarchy in the dataset. Also, the distribution of hash codes is primarily driven by semantic information that comes from the supervision labels. We propose a semantic hashing framework which utilizes the hyperbolic metric learning to learn the structural and hierarchical information. This information is leveraged in the form of proxy labels for training the hashing network with the proposed novel Structure-Semantic Disagreement (SSD) loss. SSD enforces the model to learn to hash with semantic as well as structural information, leading to more robust and uniformly distributed hash codes. Tests on multiple public domain datasets establish the effectiveness of the proposed approach. Moreover, the developed SSD loss can also be applied to other classification models to improve the representation by enforcing the model to use the structure information more effectively.","PeriodicalId":167423,"journal":{"name":"2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116788965","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Robust Watermarking for Neural Network Models Using Residual Network 基于残差网络的神经网络模型鲁棒水印
2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP) Pub Date : 2022-09-26 DOI: 10.1109/MMSP55362.2022.9949601
Lecong Wang, Zichi Wang, Xinran Li, Chuan Qin
{"title":"Robust Watermarking for Neural Network Models Using Residual Network","authors":"Lecong Wang, Zichi Wang, Xinran Li, Chuan Qin","doi":"10.1109/MMSP55362.2022.9949601","DOIUrl":"https://doi.org/10.1109/MMSP55362.2022.9949601","url":null,"abstract":"The training process of a neural network model requires plenty of costs, and so the intellectual property of neural network models should be protected. To this end, we propose a robust watermarking scheme for neural network models in this paper. In our scheme, an independent network is specially designed to help embedding watermarks into a given host network, and also be used for watermark extraction. The independent network is designed based on the residual structure which is sensitive to the parameter changes of the host network and conducive to finding suitable embedding locations. In addition, some residual blocks are randomly discarded during watermark embedding, which can increase the robustness against popular model attacks. Experimental results show that our scheme achieves satisfactory watermark verification performance without decreasing the original performance of the host network, even if the host network has been maliciously tampered.","PeriodicalId":167423,"journal":{"name":"2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)","volume":"34 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122303298","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
Secure Reversible Data Hiding in Encrypted Images based on Classification Encryption Difference 基于分类加密差异的加密图像的安全可逆数据隐藏
2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP) Pub Date : 2022-09-26 DOI: 10.1109/MMSP55362.2022.9949304
Lingfeng Qu, Hadi Amirpour, M. Ghanbari, C. Timmerer, Hongjie He
{"title":"Secure Reversible Data Hiding in Encrypted Images based on Classification Encryption Difference","authors":"Lingfeng Qu, Hadi Amirpour, M. Ghanbari, C. Timmerer, Hongjie He","doi":"10.1109/MMSP55362.2022.9949304","DOIUrl":"https://doi.org/10.1109/MMSP55362.2022.9949304","url":null,"abstract":"This paper introduces an algorithm to improve the security, efficiency, and embedding capacity of reversible data hiding in encrypted images (RDH-EI). It is based on classification encryption difference and adaptive fixed-length coding. Firstly, the prediction error image is obtained, the difference with a bin value greater than the encryption threshold in the difference histogram is found, and it is further modified to obtain the embedding threshold range. Then, under the condition of ensuring that the difference inside and outside the embedding threshold range is not confused, the difference within the threshold is only scrambled, and the difference outside the threshold is scrambled and mod encrypted. After obtaining the encrypted image, an adaptive difference fixed-length coding method is proposed to encode and compress the differences within the threshold. The secret data is embedded in the multiple most significant bits of the encoded difference. Experimental results show that the embedding capacity of the proposed algorithm is improved compared with the state-of-the-art algorithm.","PeriodicalId":167423,"journal":{"name":"2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122835018","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Hiding Function with Neural Networks 用神经网络隐藏函数
2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP) Pub Date : 2022-09-26 DOI: 10.1109/MMSP55362.2022.9949163
Yusheng Guo, Zhenxing Qian, Xinpeng Zhang
{"title":"Hiding Function with Neural Networks","authors":"Yusheng Guo, Zhenxing Qian, Xinpeng Zhang","doi":"10.1109/MMSP55362.2022.9949163","DOIUrl":"https://doi.org/10.1109/MMSP55362.2022.9949163","url":null,"abstract":"In this paper, we show that neural networks can hide a specific task while finishing a common one. We leverage the excellent fitting ability of neural networks to train two tasks simultaneously. In a classification example, the first task is normally trained by the traditional crossentropy loss. The output of the second task can be obtained by the product of the penultimate layer and a predefined random matrix. A well-trained network looks like a benign classifier and will not arouse suspicion. For certain people, they can use a predefined random matrix to turn on the second covert function of the neural network. Experimental results demonstrate that the hidden tasks can achieve satisfactory performance without affecting the original tasks.","PeriodicalId":167423,"journal":{"name":"2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)","volume":"32 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122993072","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Robust Metric Boosts Transfer 稳健的度量提高传输效率
2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP) Pub Date : 2022-09-26 DOI: 10.1109/MMSP55362.2022.9949180
Qiancheng Yang, Yong Luo, Han Hu, Xiaoxia Zhou, Bo Du, Dacheng Tao
{"title":"Robust Metric Boosts Transfer","authors":"Qiancheng Yang, Yong Luo, Han Hu, Xiaoxia Zhou, Bo Du, Dacheng Tao","doi":"10.1109/MMSP55362.2022.9949180","DOIUrl":"https://doi.org/10.1109/MMSP55362.2022.9949180","url":null,"abstract":"Transfer metric learning (TML) aims to improve the metric learning in target domains by transferring knowledge from related tasks, where the distance metrics are strong and reliable. Existing TML approaches only focus on how to transfer the source metric knowledge, which is often prone to be over-fitting to the source domain. In this paper, we study how to train a source metric that is appropriate for transfer and then design a general deep TML method for effective metric transfer. In particular, we propose to learn the source metric parameterized by a deep neural network in an adversarial way and then transfer the metric to the target domain by embedding imitation, which allows the inputs of source and target domains to be heterogeneous. Besides, we restrict the size of the target metric network to be small so that the inference is efficient in the target domain. Results in the popular face verification application demonstrate the effectiveness of our method.","PeriodicalId":167423,"journal":{"name":"2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129712120","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
As We Speak: Real-Time Visually Guided Speaker Separation and Localization 当我们说话:实时视觉引导说话人分离和定位
2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP) Pub Date : 2022-09-26 DOI: 10.1109/MMSP55362.2022.9949329
P. Czarnecki, J. Tkaczuk
{"title":"As We Speak: Real-Time Visually Guided Speaker Separation and Localization","authors":"P. Czarnecki, J. Tkaczuk","doi":"10.1109/MMSP55362.2022.9949329","DOIUrl":"https://doi.org/10.1109/MMSP55362.2022.9949329","url":null,"abstract":"Real-time speaker separation and localization is crucial to enable applications for video call enhancement, automatic subtitles localization, as well as spatial voice generation/panning. The common approach to perform speaker localization and separation is to detect candidate faces and then perform visual guided voice separation for each. There are two methods used for face detection: with face detector on static video frames [1], [2] or with audio visual sequence processing for active speaker detection [3]. In this work, we propose improvements for the visual guided speaker separation model to make it real-time. The described model follows the approach with a face detector. The model extends real-time models known for speech enhancement [4], [5] by adding face processing to ultimately perform visual guided speaker separation. Our system is lightweight with 0.6M trainable parameters. It performs speaker separation near instantaneously with the delay of a single input audio frame. To our knowledge, it is the first real-time system for visual guided speaker separation. From the application point of view it is important that the model performs both tasks at the time: speech separation and active speaker localization.","PeriodicalId":167423,"journal":{"name":"2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129157147","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Speech Enhancement Regularized by a Speaker Verification Model 基于说话人验证模型的语音增强正则化
2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP) Pub Date : 2022-09-26 DOI: 10.1109/MMSP55362.2022.9949479
Bunlong Lay, Timo Gerkmann
{"title":"Speech Enhancement Regularized by a Speaker Verification Model","authors":"Bunlong Lay, Timo Gerkmann","doi":"10.1109/MMSP55362.2022.9949479","DOIUrl":"https://doi.org/10.1109/MMSP55362.2022.9949479","url":null,"abstract":"In the last years, end-to-end neural networks were employed to enhance single-channel noisy speech. The Conv-Tasnet architecture is such a neural network and it has been successfully trained on the scale invariant signal-to-distortion ratio (SI-SDR) loss. However, we find that in a speech enhancement task Conv-Tasnet trained on the SI-SDR loss introduces distortions to the enhanced signal as the mismatch between training and testing data increases. To mitigate these effects, we investigate on a new loss function, that combines SI-SDR with a pretrained speaker verification (SV) model as a regularizer. As the SV model captures information about important characteristics of clean speech signals, we argue that the proposed regularization ensures that the estimated speech signal is speech-like even if there is an increasing mismatch between training and testing data. Accordingly, we observe considerable improvements in POLQA in mismatched training and testing conditions, e.g. when training on anechoic speech but testing on reverberant data.","PeriodicalId":167423,"journal":{"name":"2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)","volume":"93 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124665557","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Iterative Progressive-hypothesis Prediction for Forward Interframe Reconstruction of Video Compressive Sensing 视频压缩感知前向帧间重构的迭代渐进假设预测
2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP) Pub Date : 2022-09-26 DOI: 10.1109/MMSP55362.2022.9949468
Hao Liu, Renhui Sun
{"title":"Iterative Progressive-hypothesis Prediction for Forward Interframe Reconstruction of Video Compressive Sensing","authors":"Hao Liu, Renhui Sun","doi":"10.1109/MMSP55362.2022.9949468","DOIUrl":"https://doi.org/10.1109/MMSP55362.2022.9949468","url":null,"abstract":"For the forward interframe reconstruction of video compressive sensing, the existing algorithms derive their residual sparse models by multihypothesis prediction from a forward keyframe or a previous non-keyframe. However, these algorithms simply fix the quantity of hypothesis candidates during successive approximation of each non-keyframe. Therefore, this paper proposes an iterative progressive-hypothesis prediction algorithm which provides a hypothesis descent prior for the multihypothesis prediction. The proposed algorithm alternately implements the prior-guided patch matching and prediction-based residual sparse modelling to improve the recovery quality of each non-keyframe. By introducing a coarse-to-fine hypothesis selection mechanism, the proposed algorithm can obtain higher matching accuracy for the interframe multihypothesis. As compared with other interframe reconstruction algorithms, the proposed algorithm can achieve good recovery quality under the low-delay reconstruction framework.","PeriodicalId":167423,"journal":{"name":"2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130629774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Multi-modal Human-machine Conversation System for Real Physical World 面向真实物理世界的多模态人机对话系统
2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP) Pub Date : 2022-09-26 DOI: 10.1109/MMSP55362.2022.9949120
Yitao Chen, Shibo Nie, Mandan Guan, Jie Wang, Ruoyi Du, Dongliang Chang, K. Liang, Zhanyu Ma
{"title":"Multi-modal Human-machine Conversation System for Real Physical World","authors":"Yitao Chen, Shibo Nie, Mandan Guan, Jie Wang, Ruoyi Du, Dongliang Chang, K. Liang, Zhanyu Ma","doi":"10.1109/MMSP55362.2022.9949120","DOIUrl":"https://doi.org/10.1109/MMSP55362.2022.9949120","url":null,"abstract":"Enabling machines to process multi-modal information and understand the real physical world is an important step to achieving free human-machine conversation. However, previous human-machine conversation systems are mostly limited to single-modal (e.g., chat robot), single-round (e.g., visual Q & A), and static visual information (e.g., visual dialogue). To address the above problem, we develop a multi-modal human-machine conversation system for specific application scenarios. The system includes two modules: (i) an interactive visual grounding module that can actively disambiguate user's queries, and (ii) an interactive fine-grained recognition module that can model objects in the 3D environment and actively ask for missing visual information. A video demo of our system under the automobile sales scenario can be found here11https://drive.google.com/file/d/1IfBsMKq55ryLOZchIT6G-R5CyWnC3Skiew?usp=sharing.","PeriodicalId":167423,"journal":{"name":"2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)","volume":"109 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124238976","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Learning-Based Multi-Stage Intra Partition for Versatile Video Coding 基于学习的多阶段内分割多用途视频编码
2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP) Pub Date : 2022-09-26 DOI: 10.1109/MMSP55362.2022.9949313
Hongji Zeng, Tiesong Zhao, Weize Feng, Nan Chen, Jielian Lin, Xu Wang
{"title":"Learning-Based Multi-Stage Intra Partition for Versatile Video Coding","authors":"Hongji Zeng, Tiesong Zhao, Weize Feng, Nan Chen, Jielian Lin, Xu Wang","doi":"10.1109/MMSP55362.2022.9949313","DOIUrl":"https://doi.org/10.1109/MMSP55362.2022.9949313","url":null,"abstract":"The latest standard, Versatile Video Coding (VVC), doubles the coding efficiency over the previous generation standard. However, better performance is at the cost of a sharp increase in coding complexity. In order to reduce the complexity of VVC intra coding, this paper proposes a multi-stage block partition decision framework based on deep learning. First, we propose a three-stage redundant modes removal framework that decreases the number of modes checked in the brute-force process. Then, we build a lightweight CNN to complete the classification task of each stage. To reduce the burden of CNN and adapt to different Coding Unit (CU) sizes, we pre-process the luminance component of CU and use the results as input of the network. Finally, the multi-threshold adjusting scheme is proposed for trading off complexity reduction with the bit-rate increase. The experimental results shows our method can reduce the encoding time ranging from 16.93% to 69.40% with the bit-rate increase ranging from 0.31% to 3.59%. Such results demonstrate that our method has superior performance with a wide range of adjustments compared with other state-of-the-art methods.","PeriodicalId":167423,"journal":{"name":"2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP)","volume":"114 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-09-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123345335","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 1
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书