Yu Zhao;Xiaofeng Cong;Chuanrui Hu;Lu Dong;Jie Gui;James Tin-Yau Kwok;Yuyan Yan Tang
{"title":"Learning From the Abyss: Self-Adaptive Distillation for Enhancing Underwater Images With Complex Degradations","authors":"Yu Zhao;Xiaofeng Cong;Chuanrui Hu;Lu Dong;Jie Gui;James Tin-Yau Kwok;Yuyan Yan Tang","doi":"10.1109/TMM.2026.3689437","DOIUrl":"https://doi.org/10.1109/TMM.2026.3689437","url":null,"abstract":"While deep learning has advanced underwater image enhancement (UIE), progress is fundamentally hampered by the scarcity of paired training data. Semi-supervised methods, which leverage abundant unlabeled images, offer a promising solution but often employ naive, static weighting for pseudo-labels. This overlooks a key principle: the most informative training samples are those that represent a successful restoration from severe degradation, thus possessing a greater transformation difficulty. To address this, we propose a novel adaptive training paradigm, the Self-Distilled Degradation-Aware Network (SDDA-Net), governed by an incentive-driven weighting mechanism. We introduce a scoring function that evaluates each pseudo-label by considering both its perceptual quality and a term that is inversely proportional to its structural similarity to the input. This score, which quantifies the difficulty and success of the enhancement, is then used to dynamically modulate the sample’s influence on the unsupervised loss. By allocating greater learning weights to successfully restored, complex examples, our framework forces the model to focus on mastering challenging degradations. Extensive experiments on multiple benchmark datasets demonstrate that this adaptive approach significantly outperforms current state-of-the-art methods, validating that the strategic, sample-adaptive modulation of the learning process is critical for success in data-scarce restoration tasks.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"9269-9280"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149130693","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Hierarchical Structure Consistency Learning for Multimodal Emotion Recognition","authors":"Boan Yang;Qinghongya Shi;Xiaofen Zong;Mang Ye","doi":"10.1109/TMM.2026.3694526","DOIUrl":"https://doi.org/10.1109/TMM.2026.3694526","url":null,"abstract":"Multimodal Emotion Recognition (MER) aims to infer emotions from multimodal data. Existing studies have leveraged attention mechanisms and modal fusion strategies with significant success in emotional understanding. However, these methods struggle with inadequate intra-modal interaction and modal asynchrony. This study introduces an innovative Hierarchical Structure Consistency Learning method for Multimodal Emotion Recognition (HSCL) to tackle these issues. As for inadequate intra-modal interaction, HSCL designs a semantic consistency algorithm with a margin constraint to enhance modal aggregation. This approach focuses on strengthening intra-modal interactions and addressing the inadequacies in emotional context comprehension, thereby deepening the understanding within each modality. For modal asynchrony, HSCL optimizes representation consistency at the modality level and geometric consistency at the sample level, addressing modal asynchrony and improving model performance. In addition to applying our structure consistency on the original multimodal features, our HSCL computes reconstructed features from decoupled representations and performs hierarchical consistency calculations on these reconstructed features. This dual application mitigates the potential negative impact of the training process on consistency, thereby enhancing adaptability and performance.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"9623-9637"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149130832","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Photovoltaic Defect Image Generator With Boundary Alignment Smoothing Constraint for Domain Shift Mitigation","authors":"Dongying Li;Binyi Su;Hua Zhang;Yong Li;Haiyong Chen","doi":"10.1109/TMM.2026.3679929","DOIUrl":"https://doi.org/10.1109/TMM.2026.3679929","url":null,"abstract":"Accurate defect detection of photovoltaic (PV) cells is critical for ensuring quality and efficiency in intelligent PV manufacturing systems. However, the scarcity of rich defect data poses substantial challenges for effective model training. While existing methods have explored generative models to augment datasets, they often suffer from instability, limited diversity, and domain shifts. To address these issues, we propose PDIG, a <underline>P</u>hotovoltaic <underline>D</u>efect <underline>I</u>mage <underline>G</u>enerator based on Stable Diffusion (SD). PDIG leverages the strong priors learned from large-scale datasets to enhance generation quality under limited data. Specifically, we introduce a <underline>S</u>emantic <underline>C</u>oncept <underline>E</u>mbedding (SCE) module that incorporates text-conditioned priors to capture the relational concepts between defect types and their appearances. To further enrich the domain distribution, we design a <underline>L</u>ightweight <underline>I</u>ndustrial <underline>S</u>tyle <underline>A</u>daptor (LISA), which injects industrial defect characteristics into the SD model through cross-disentangled attention. At inference, we propose a <underline>T</u>ext-<underline>I</u>mage <underline>D</u>ual-<underline>S</u>pace <underline>C</u>onstraints (TIDSC) module, enforcing the quality of generated images via positional consistency and spatial smoothing alignment. Extensive experiments demonstrate that PDIG achieves superior realism and diversity compared to state-of-the-art methods. Specifically, our approach improves Frechet Inception Distance (FID) by 19.16 points over the second-best method and significantly enhances the performance of downstream defect detection tasks.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"8481-8493"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149130899","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"RCOC-Distill: Boosting 4D Radar-Camera Online Calibration With Knowledge Distillation From LiDAR Features","authors":"Long Zhuang;Yiqing Yao;Tao Zhang","doi":"10.1109/TMM.2026.3673505","DOIUrl":"https://doi.org/10.1109/TMM.2026.3673505","url":null,"abstract":"Extrinsic calibration of 4D millimeter-wave radar and camera is a prerequisite for multi-sensor fusion in complex environments. Online calibration algorithms can dynamically correct jitter errors caused by vibrations and have increasingly become the predominant approach. However, online calibration for 4D radar-camera systems is constrained by the sparsity of radar point clouds, making it challenging to establish feature correspondences between radar and camera. This paper proposes a novel 4D radar-camera online calibration algorithm, RCOC-Distill, which leverages cross-modal knowledge distillation (CMKD) to enhance 4D radar representations with LiDAR-derived features, thereby improving calibration accuracy. A Local Correlation Layer (LCL) is introduced to compute feature similarity within a local window via convolution operations, reducing computational burden while maintaining local consistency. Additionally, a Radar-LiDAR Cross-Modal Alignment (RLCMA) module is proposed to further enhance radar feature density, mitigating inefficiencies in knowledge transfer from LiDAR to 4D radar. The proposed method is evaluated on two datasets (NTU4DRaDLM and Dual-Radar). Compared to state-of-the-art models, RCOC-Distill achieves improvements of 38.73% and 23.48% in translation and rotation errors, respectively, on the Dual-Radar dataset. These results validate the superiority of the proposed method.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"7936-7949"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149131171","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Gai Zhang;Xinfeng Zhang;Lv Tang;Hongyu An;Li Zhang;Qingming Huang
{"title":"SANR: Scene-Aware Neural Representation for Light Field Image Compression With Rate-Distortion Optimization","authors":"Gai Zhang;Xinfeng Zhang;Lv Tang;Hongyu An;Li Zhang;Qingming Huang","doi":"10.1109/TMM.2026.3686627","DOIUrl":"https://doi.org/10.1109/TMM.2026.3686627","url":null,"abstract":"Light field images capture multi-view scene information and play a crucial role in 3D scene reconstruction. However, their high-dimensional nature results in enormous data volumes, posing a significant challenge for efficient compression in practical storage and transmission scenarios. Although neural representation-based methods have shown promise in light field image compression, most approaches rely on direct coordinate-to-pixel mapping through implicit neural representation (INR), often neglecting the explicit modeling of scene structure. Moreover, they typically lack end-to-end rate-distortion optimization, limiting their compression efficiency. To address these limitations, we propose SANR, a Scene-Aware Neural Representation framework for light field image compression with end-to-end rate-distortion optimization. For scene awareness, SANR introduces a hierarchical scene modeling block that leverages multi-scale latent codes to capture intrinsic scene structures, thereby reducing the information gap between INR input coordinates and the target light field image. From a compression perspective, SANR is the first to incorporate entropy-constrained quantization-aware training (QAT) into neural representation-based light field image compression, enabling end-to-end rate-distortion optimization. Extensive experiment results demonstrate that SANR significantly outperforms state-of-the-art techniques regarding rate-distortion performance with a 65.62% BD-rate saving against HEVC.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"9321-9332"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149131217","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"AS-RAM: Automated Search With Rich Attention Modules for Network Traffic Prediction","authors":"Jongkyung Im;Sung Oh;Myungjun Oh;Ji-yeon Park;Joung-Sik Kim;Na-rae Yi;Myungho Kim;Sung-Ho Bae","doi":"10.1109/TMM.2026.3684751","DOIUrl":"https://doi.org/10.1109/TMM.2026.3684751","url":null,"abstract":"By effectively forecasting network traffic patterns, future cellular networks have the potential to manage themselves and incorporate intelligent, streamlined operations autonomously. Existing methods have relied on handcrafted architectures to predict network traffic, which still have two main limitations. First, human-designed models may have sub-optimality in modeling. Second, existing handcrafted designs did not take spatio-temporal long-term dependencies into account explicitly even though they play crucial roles in network traffic prediction. To address these issues, we propose a method using NAS (Neural Architecture Search) to find more optimized architectures than handcrafted ones for network traffic prediction. Specifically, we introduce a new search space that integrates attention modules enabling better extraction and reflection of the unique characteristics of network traffic data. These attention modules are seamlessly incorporated into the architecture from an optimization perspective. To verify the effectiveness of the proposed method, we validate the performance of the proposed framework using the Italia Telecom dataset. Experimental results show that our framework significantly improves the performance compared to the baseline in terms of root mean squared error between predicted value and ground-truth value. Specifically, our method improves the performance by 9.5%, 9.8%, and 10.2% on SMS, CALL, and INTERNET traffic datasets, respectively.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"8864-8875"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149131351","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Empowering Non-Parametric Network With Intrinsic Attention and Optimal Transport for Few-Shot 3D Segmentation","authors":"Jiacheng Xu;Ying Li;Bin Tan;Die Hu;Jun Wu","doi":"10.1109/TMM.2026.3699028","DOIUrl":"https://doi.org/10.1109/TMM.2026.3699028","url":null,"abstract":"Current approaches to few-shot 3D point cloud segmentation typically depend on heavily pre-trained and parameterized networks. These methods tend to introduce substantial computational costs and exhibit limited generalization ability when applied to new categories across different domains. To address these challenges, we propose a novel Non-Parametric network for few-shot 3D scene Segmentation, NP-3DS. Unlike existing methods that use fixed fusion schemes, our approach proposes a non-parametric intrinsic attention aggregation mechanism that adaptively integrates spatial and chromatic features, effectively preserving fine-grained structures in complex scenes. To mitigate prototype bias and improve cross-domain transferability, we propose an optimal transport prototype calibration strategy that dynamically aligns category representations between support and query sets. Furthermore, to restore structural fidelity during feature upsampling, we design a volume-inverse weighted trilinear interpolation method that captures local geometric correlations, enabling accurate recovery of sharp edges and fine details. Experiments on two benchmarks (S3DIS and ScanNet) demonstrate that NP-3DS significantly outperforms the state-of-the-art non-parametric method (gains +7.37% and +4.48% mIoU, respectively) and matches or surpasses parametric baselines. These results highlight the efficiency and generalizability of our method in a few-shot 3D scene segmentation.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"9896-9907"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149131352","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Xiaozhao Fang;Yuanyuan Liu;Yonghao Chen;Na Han;Weijun Sun
{"title":"Central Similarity Representation Learning for Efficient Cross-Domain Retrieval","authors":"Xiaozhao Fang;Yuanyuan Liu;Yonghao Chen;Na Han;Weijun Sun","doi":"10.1109/TMM.2026.3682157","DOIUrl":"https://doi.org/10.1109/TMM.2026.3682157","url":null,"abstract":"To better improve the retrieval accuracy by transferring knowledge from the source domain to the target domain, cross-domain problem has recently been considered in image retrieval, i.e., cross-domain retrieval. Current approaches appropriately combine domain adaptation technology with learning to hash, achieving some successes. However, they mainly perform local similarity preserving or conditional distribution alignment as a solution to accurately retrieve cross-domain data, which poses the following limitations: 1) They ignore the data-to-category and category-to-category relationships; 2) Existing methods incur excessive computational expenses for a retrieval task. To address these limitations, we introduce an efficient cross-domain retrieval approach called <inline-formula><tex-math>$mathbf {C}$</tex-math></inline-formula>entral <inline-formula><tex-math>$mathbf {S}$</tex-math></inline-formula>imilarity <inline-formula><tex-math>$mathbf {R}$</tex-math></inline-formula>epresentation <inline-formula><tex-math>$mathbf {L}$</tex-math></inline-formula>earning (<inline-formula><tex-math>$mathbf {CSRL}$</tex-math></inline-formula>) for efficient cross-domain retrieval. Firstly, the marginal distribution alignment is employed to obtain a domain-agnostic space. Then we perform a novel central similarity learning for category alignment. Hence, discriminative enhancement is achieved by capturing data-to-category and category-to-category relationships. Secondly, we transform heterogeneous data into domain-agnostic semantic representations, and a common semantic space is also obtained. Specifically, guided by labels, discriminability is transferred to the Hamming space through semantic representation learning. Furthermore, our method incurs lower computational cost. Experimental results on standard databases demonstrate that the CSRL achieves state-of-the-art performance.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"8557-8570"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149131530","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"D$^{2}$Net: Diagonal-Guided Dual-Stream Network Based on Wavelet Transform for Image Demoiréing","authors":"Junhao Tan;Songwen Pei","doi":"10.1109/TMM.2026.3684307","DOIUrl":"https://doi.org/10.1109/TMM.2026.3684307","url":null,"abstract":"Moiré patterns in images significantly hinder the capture of multimedia content on the screen using mobile devices, introducing pronounced color degradation, deviation, and texture occlusion. Existing RAW- and RGB-based methods are inherently constrained by the frequency-domain texture limitations imposed by image encoding formats, leading to incomplete moiré suppression. Although existing wavelet-based methods enrich frequency information, they typically process high-frequency sub-bands uniformly without considering their distinct roles in moiré perception, and rely on heuristic convolutional stacking without principled interpretability or motivation. To address this, our analysis reveals that moiré perception diverges across high-frequency sub-bands, with diagonal frequencies being the least affected. Based on this insight, we propose a Diagonal-Guided Dual-Stream Network (D<inline-formula><tex-math>$^{2}$</tex-math></inline-formula>Net), leveraging a divide-and-conquer paradigm. In the low-frequency stream, we introduce a Domain Refinement Module (DRM) to mitigate color moiré-induced deviations while consolidating multi-scale channel features. In the high-frequency stream, we propose a Diagonal-Guided Attention Block (DGAB), which utilizes diagonal frequency sub-bands to guide vertical and horizontal frequency sub-bands for moiré texture elimination. Both quantitative and qualitative experiments demonstrate that our model achieves superior performance while maintaining exceptional parameter efficiency.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"8722-8733"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149131638","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Motion-Aware Representation Network for Micro-Expression Recognition","authors":"Shiting Xu;C. L. Philip Chen;Tong Zhang","doi":"10.1109/TMM.2026.3675939","DOIUrl":"https://doi.org/10.1109/TMM.2026.3675939","url":null,"abstract":"Micro-expression (ME) are brief, involuntary facial motions that reveal genuine emotions. Their low intensity and short duration pose challenges for Micro-expression Recognition (MER). However, robust and precise motion representations are crucial for improving MER performance. In this paper, we propose a Motion-Aware Representation Network (MARNet). Specifically, MARNet learns feature-level motion representations from the onset and apex frames of ME samples. It innovatively learns gradient information from features to learn highly sensitive and localized ME representations. To improve data utilization and facilitate representation learning, WarpingNet is proposed using target frame reconstruction as a self-supervised guidance signal. We further design motion collaborative learning to improve motion representation correctness and emotion discriminability by introducing the explicit effect of optical flow with label-aware contrastive learning. Extensive experiments show that our proposed method outperforms the state-of-the-art method on four datasets.","PeriodicalId":13273,"journal":{"name":"IEEE Transactions on Multimedia","volume":"28 ","pages":"8597-8607"},"PeriodicalIF":9.9,"publicationDate":"2026-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149131761","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}