IEEE Transactions on Multimedia最新文献

{"title":"Reversible Data Hiding in Encrypted Images With Hybrid Encoding and Bitstream Rearrangement","authors":"Lizhi Xiong;Yunfei Ji;Ching-Nung Yang;Zhangjie Fu","doi":"10.1109/TMM.2026.3673524","DOIUrl":"https://doi.org/10.1109/TMM.2026.3673524","url":null,"abstract":"Reversible data hiding in encrypted images (RDHEI) has become increasingly important for cloud storage and privacy protection. However, striking a balance between embedding capacity and security remains a significant challenge for researchers. This paper proposes a hybrid Encoding RDHEI scheme that combines custom mapping embedding (CME) and high bit-plane replacement (HBR) methods. By employing a hybrid encoding technique that encodes prediction errors (PEs) of different sizes, the proposed scheme significantly improves the number of embeddable pixels. Unlike existing RDHEI schemes, the proposed scheme embeds label information into a reserved space during the preprocessing and then encrypts it, thereby offering enhanced security during image transmission. Experimental results and analysis demonstrate that our scheme outperforms similar existing schemes in terms of embedding performance, visual quality, stability, and computational complexity. Our proposed scheme achieves a superior performance in a multi-metric evaluation.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"7554-7567"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148837370","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"ExpDiff: Generating High-Fidelity 3D Facial Expression Meshes and BRDF Textures via Diffusion Model","authors":"Yuhao Cheng;Xuanchen Li;Xingyu Ren;Zhuo Chen;Chenghui Ke;Xiaokang Yang;Yichao Yan","doi":"10.1109/TMM.2026.3668519","DOIUrl":"https://doi.org/10.1109/TMM.2026.3668519","url":null,"abstract":"3D face generation is a critical task for immersive multimedia applications, where a key challenge is the joint synthesis of expressive geometry and BRDF textures. Existing methods often struggle with geometric-textural coherence and corresponding reflectance modeling. To overcome these limitations, we present ExpDiff, a framework that generates expression meshes and corresponding BRDF textures from a single neutral-expression face. Our method employs an attention-based diffusion model to learn the semantic transition across expressions. To ensure correspondence between geometry and texture, we introduce a unified representation that explicitly models geometric-textural interaction, which is encoded into a shared latent space by models pre-trained on a vast dataset for strong generalization. To achieve semantically coherent and physically consistent generation, we propose to guide the denoising direction with specially designed textual prompts. We further construct two novel facial expression datasets, J-Reflectance, for ultra-high-quality assets, and FFHQ-BRDFExp for diverse identities, both of which are publicly released to advance the community. Extensive experiments demonstrate our method's superior performance in photo-realistic facial expression synthesis.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"5863-5875"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148487336","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Toward Bidirectional Adaptability for Few-Shot Class-Incremental Learning With Forward-Backward Knowledge Transfer","authors":"Bingzhi Chen;Zhiming Chen;Sudong Cai;Xiaozhao Fang;Mohammed Bennamoun;Shengli Xie","doi":"10.1109/TMM.2026.3651015","DOIUrl":"https://doi.org/10.1109/TMM.2026.3651015","url":null,"abstract":"The development of Deep Neural Networks (DNNs) has enabled AI-driven models to excel in recognizing a limited set of classes within static environments. As AI systems progress, few-shot class-incremental learning (FSCIL) aims to expand their understanding of novel classes from minimal samples while retaining knowledge of previously encountered ones. However, most existing FSCIL models face significant challenges, including <italic>inadequate adaptability</i> and <italic>catastrophic forgetting</i>, which hinder their ability to maintain robust forward and backward learning capabilities. To address these issues, this paper proposes a novel Forward-Backward Knowledge Transfer (FBKT) paradigm¹, which strategically integrates forward distribution adaptation (FDA) and backward semantic alignment (BSA) mechanisms to achieve bidirectional adaptability in knowledge transfer. The FDA mechanism enhances forward adaptability by expanding and reserving the embedding space for new classes using semantic-irrelevant masked images as virtual negative classes, thereby mitigating data overfitting. It also employs self-supervised representation learning to utilize semantic-relevant local embeddings as additional positive samples, fostering class separation and generalization. Meanwhile, the BSA mechanism ensures the semantic consistency of previously learned classes across sessions during class-incremental learning, promoting smoother backward adaptability and reducing model degradation. Extensive experiments conducted on multiple benchmark datasets consistently highlight the superior performance and effectiveness of our FBKT compared to state-of-the-art methods.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"2337-2351"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148488242","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"SOI-Net: Structural Optimization-Inspired Interpretable Network for Incomplete Multi-View Clustering","authors":"Chunming Wu;Zihan Fang;Zhiling Cai;Shide Du;Wei Huang;Jinbo Li;Shiping Wang","doi":"10.1109/TMM.2025.3623519","DOIUrl":"https://doi.org/10.1109/TMM.2025.3623519","url":null,"abstract":"Data missing is a common issue in real-world applications, posing significant challenges for incomplete data processing. Traditional incomplete multi-view clustering methods rely on manually-designed optimization problems based on prior interpretable knowledge, considering the full utilization of available data. However, their limited feature extraction capability may become a bottleneck. In contrast, deep optimization methods leverage learning-based nonlinear transformations for clustering. They primarily achieve data imputation through the generalization ability of deep models, but their model interpretability may be limited by the black-box nature. Moreover, most existing methods only explore the structure of each view independently, where these structures are fixed and cannot form a complete unified structure. To address these issues, we propose a Structural Optimization-inspired Interpretable Network (SOI-Net) for incomplete multi-view clustering. Specifically, we project the features of all views into a unified representation space with the un-missing information of the views as constraints. By optimizing consistent structural information, we preserve the structures of missing modalities in the unified representation space, thereby mitigating the impact of missing data. Meanwhile, we derive network components based on the optimization problem to guide the learning of structure and representation. The practical significance of these network components provides model design-level interpretability. Extensive experiments on six datasets validate the effectiveness of SOI-Net in handling incomplete multi-view clustering task.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"196-208"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148492424","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Auto-DBPA: Density-Aware Ball-Pivoting Algorithm With Adaptive Radius Using Contextual Bandits for Object and Scene Reconstruction","authors":"Houda Saffi;Naima Otberdout;Youssef Hmamouche;Amal El Fallah Seghrouchni","doi":"10.1109/TMM.2025.3623546","DOIUrl":"https://doi.org/10.1109/TMM.2025.3623546","url":null,"abstract":"The Ball-Pivoting Algorithm (BPA) is a crucial technique for 3D surface reconstruction from point clouds, which relies heavily on selecting an appropriate ball radius. The effectiveness of BPA is significantly influenced by the point sampling density. In areas of low sampling density, a small ball radius can create gaps, while a large radius in high-density regions may oversimplify the surface and miss finer details. This paper addresses this challenge by introducing Auto-DBPA, an Automatic radius selection with Density-aware BPA. Auto-DBPA adapts dynamically by adjusting the ball radius according to the local sampling density. Our approach offers a scalable solution for reconstructing complex scenes and objects with varying levels of detail. Unlike conventional methods that require partitioning the point cloud into clusters and merging the reconstructed parts, which can cause visible seams and discontinuities, our unified reconstruction pipeline dynamically adjusts radii across the entire point cloud. To achieve this, we use hierarchical clustering and compute Fast Point Feature Histograms (FPFH) for each density cluster, capturing local geometric properties. We then leverage these geometric features to predict the optimal radius values for each cluster, adequately adapting the ball radius to local density variations. We also address the non-differentiability of BPA, which arises from its geometric computations and lack of gradient information, by introducing an innovative solution based on contextual bandits. Our approach employs the contextual bandits framework to effectively select the optimal ball radius based on local geometric features, significantly enhancing reconstruction quality. Our method is scalable and particularly effective in scenarios with varying density levels where a single-radius solution is inadequate. Results on the ABC, FAUST, SceneNN and ScanNet datasets demonstrate that our method successfully handles 3D reconstruction from varying point cloud densities, outperforming manual tuning, classic methods, and learning-based approaches.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"643-656"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148492961","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"High-Frequency Prioritized Sparse Attention Network for Image Restoration","authors":"Shuting Dong;Zhe Wu;Hongyang Wei;Mingzhi Chen;Guanghao Li;Haolong Qian;Hanyang Peng;Chun Yuan","doi":"10.1109/TMM.2025.3618558","DOIUrl":"https://doi.org/10.1109/TMM.2025.3618558","url":null,"abstract":"Image restoration aims to restore high-quality images from degraded inputs caused by factors such as motion blur, defocus blur, and rain, where the primary difference between degraded and high-quality images lies in their high-frequency components. Despite the critical role of high frequencies in restoration, few methods explicitly prioritize computational resources for high frequencies over low frequencies. To address this issue, we propose a High-Frequency Prioritized Sparse Attention Network (HFP-SAN), a novel architecture for image restoration tasks. We explicitly prioritize high-frequency components by designing a symmetric encoder-decoder framework integrated with High-Frequency Selective Sparse Attention (HFSSA) modules while handling low-frequency components with a smaller residual network, thereby proportionally allocating computational resources based on their relative importance. HFSSA incorporates a Frequency-Selective Matching (FSM) algorithm to focus attention on strongly correlated high-frequency regions, mitigating computation on areas with weak correlations and irrelevant areas. Additionally, we introduce a dynamically adjustable high-frequency mask that guides the network to focus on the severely degraded regions, further refining restoration quality. The above designs ensure the final reconstructed image is a high-quality product. Experiments demonstrate that our HFP-SAN achieves state-of-the-art performance across multiple image restoration tasks, both quantitatively and qualitatively.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"3458-3470"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148487866","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Novel Robust Reversible Watermarking Scheme Using Fractional-Order Polar Complex Exponential Transform","authors":"Yichao Tang;Shuai Wang;Ruowei Han;Chuntao Wang","doi":"10.1109/TMM.2025.3623522","DOIUrl":"https://doi.org/10.1109/TMM.2025.3623522","url":null,"abstract":"Robust reversible watermarking (RRW) techniques have been proposed in the literature to protect the copyrights of high-fidelity digital images while achieving robustness, reversibility, invisibility, and large capacity simultaneously. Most studies on RRW have been designed to resist common signal processing (CSP) attacks, but only a few can withstand both CSP and geometric deformation (GD) attacks. To address this problem, this study proposes a novel RRW method using a fractional-order polar complex exponential transform (FrPCET) and optimized quantization index modulation (QIM). Specifically, the optimal fractional parameter of the FrPCET is determined through numerical simulation experiments using the criterion of minimum image reconstruction errors. The stability of FrPCET moments against CSP is evaluated by performing attack simulation tests on 500 images, revealing that differences between specific pairs of FrPCET moments exhibits similar variation patterns under attacks, thus making them suitable for use as embedding carriers. Then, the watermark is embedded by optimizing a conventional QIM, which improves the robustness of the watermark under the same image quality conditions. The distortions caused by watermark embedding and the hash sequences used for integrity authentication are subsequently taken as the auxiliary information and are reversibly embedded via the prediction error expansion-histogram shift method. After receiving the watermarked image, the receiver performs an inverse operation to recover both the watermark and the original image in the absence of attacks; otherwise, it only extracts the watermark. Extensive simulation experiments demonstrate that the proposed method has greater robustness against various CSP and GD attacks than do state-of-the-art methods under the same embedding capacity and invisibility. This indicates the feasibility and effectiveness of the proposed scheme.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"241-255"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148489420","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Twin Tensor Learning for Consistency and Inconsistency: A Unified Affinity Learning Framework for Multi-View Clustering","authors":"Zhibin Gu;Songhe Feng","doi":"10.1109/TMM.2026.3660113","DOIUrl":"https://doi.org/10.1109/TMM.2026.3660113","url":null,"abstract":"Due to the efficiency of discovering the high-order correlations of multiple views, tensorized multi-view clustering (TMVC) has been extensively investigated and achieve impressive performance. However, previous TMVC studies merely emphasize on exploring the multi-view consistency, but rarely consider the cross-view inconsistency that may be caused by noise, corruptions or view-specific properties, rendering the abundant information contained in multiple features cannot be comprehensively detected. To this end, this paper proposes a <bold>T</b>win tens<bold>O</b>rized affinity learning framew<bold>O</b>rk for mul<bold>T</b>i-view clu<bold>S</b>tering (<bold>TOOTS</b>), which seamlessly integrates multi-view consistency and cross-view inconsistency into a unified framework. Specifically, the TOOTS model employs the self-representation mechanism to learn two sets of affinity matrices, which are reorganized into twin tensors (i.e., consistent tensor and diverse tensor). Meanwhile, tensor low-rank constraint and exclusive lasso with <inline-formula><tex-math>$ell _{21}$</tex-math></inline-formula> regularization are imposed on consistent part and diverse part respectively, leading to the intra-view spatial structure and the inter-view inconsistency be exploited simultaneously. Furthermore, the multiple consistent factors are manipulated by a connectivity constraint and fused into a consensus graph with different weights, making the explicit clusters structural and the salient difference between views effectively leveraged. Substantial experimental results demonstrate that the proposed TOOTS model outperforms state-of-the-arts.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"4233-4244"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148490910","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Isharah: A Large-Scale Multi-Scene Dataset for Continuous Sign Language Recognition","authors":"Sarah Alyami;Hamzah Luqman;Sadam Al-Azani;Maad Alowaifeer;Yazeed Alharbi;Yaser Alonaizan","doi":"10.1109/TMM.2026.3664959","DOIUrl":"https://doi.org/10.1109/TMM.2026.3664959","url":null,"abstract":"Current benchmarks for sign language recognition (SLR) focus mainly on isolated SLR, while there are limited datasets for continuous SLR (CSLR), which recognizes sequences of signs in a video. Additionally, existing CSLR datasets are collected in controlled settings, which restricts their effectiveness in building robust real-world CSLR systems. To address these limitations, we present Isharah, a large multi-scene dataset for CSLR. It is the first dataset of its type and size, collected in an unconstrained environment using signers’ smartphones. This setup resulted in high variations of recording settings, camera distances, angles, and resolutions. This variation helps with developing sign language understanding models capable of handling the variability and complexity of real-world scenarios. The dataset consists of 30,000 video clips performed by 18 deaf and professional signers. Additionally, the dataset is linguistically rich as it provides a gloss-level annotation for all dataset’s videos, making it useful for developing CSLR and sign language translation (SLT) systems. This paper also introduces multiple sign language understanding benchmarks, including signer-independent and unseen-sentence CSLR, along with gloss-based and gloss-free SLT.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"5050-5058"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148488790","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Visual Context and Commonsense-Guided Causal Chain-of-Thoughts for Visual Commonsense Reasoning","authors":"Xinyu Li;Jing Zhao;Tongquan Wei;Shiliang Sun","doi":"10.1109/TMM.2026.3651070","DOIUrl":"https://doi.org/10.1109/TMM.2026.3651070","url":null,"abstract":"Humans are capable of inferring dynamic context from a still image and, with the provision of additional commonsense knowledge, can accurately complete visual commonsense reasoning tasks. Nevertheless, this remains a highly challenging cognitive-level task for current vision-language models. Previous work has primarily focused on utilizing models fine-tuned for specific downstream tasks and introduces external world knowledge to tackle these challenging tasks, while neglecting the importance of accurate context and the key role of commonsense knowledge in reasoning. In this paper, we propose a novel framework to enhance visual commonsense reasoning by incorporating context and commonsense knowledge. We decompose the visual commonsense reasoning problem into four distinct but interrelated sub-problems and combine visual language models with a large language model to enable zero-shot reasoning. The uniqueness of this work lies in the proposed commonsense knowledge filtering module, which filters out relevant commonsense knowledge through the causal strength of visual context. This process constructs Visual Context and Commonsense-guided Causal Chain-of-Thought (<inline-formula><tex-math>$mathrm{VC^{3}}$</tex-math></inline-formula>-CoT) reasoning paths, thereby providing double robustness to visual commonsense reasoning by incorporating weighted majority voting strategy. Extensive experiments on several downstream tasks demonstrate that the proposed method significantly improves performance compared to baseline models and the state-of-the-art method, and confirm the effectiveness of the proposed components.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"2719-2730"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148488795","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}