Journal of Visual Communication and Image Representation最新文献

{"title":"MTPA: A multi-aspects perception assisted AIGV quality assessment model","authors":"Yun Liu,&nbsp;Daoxin Fan,&nbsp;Zihan Liu,&nbsp;Sifan Li,&nbsp;Haiyuan Wang","doi":"10.1016/j.jvcir.2026.104721","DOIUrl":"10.1016/j.jvcir.2026.104721","url":null,"abstract":"<div><div>With the development of Artificial Intelligence (AI) generated technology, AI generated video (AIGV) has aroused much attention. Compared to the visual perceptual in traditional video, AIGV has its unique challenges, such as visual consistency, text-to-video alignment, etc. In this paper, we propose a multi-aspect perception assisted AIGV quality assessment model, which gives a comprehensive quality evaluation of AIGV from three aspects: text–video alignment score, visual spatial perceptual score, and visual temporal perceptual score. Specifically, a pre-trained vision-language module is adopted to study the text-to-video alignment quality, and the semantic-aware module is applied to capture the visual spatial perceptual features. Besides, an effective visual temporal feature extraction module is used to capture multi-scale temporal features. Finally, text–video alignment features, visual spatial, visual temporal perceptual features, and multi-scale visual fusion features are integrated to give a comprehensive quality evaluation. Our model holds state-of-the-art results on three public AIGV datasets, proving its effectiveness.</div></div>","PeriodicalId":54755,"journal":{"name":"Journal of Visual Communication and Image Representation","volume":"116 ","pages":"Article 104721"},"PeriodicalIF":3.1,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146024842","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multimodal prompt-guided vision transformer for precise image manipulation localization","authors":"Yafang Xiao ,&nbsp;Wei Jiang ,&nbsp;Shihua Zhou ,&nbsp;Bin Wang ,&nbsp;Pengfei Wang ,&nbsp;Pan Zheng","doi":"10.1016/j.jvcir.2026.104736","DOIUrl":"10.1016/j.jvcir.2026.104736","url":null,"abstract":"<div><div>With the rise of generative AI and advanced image editing technologies, image manipulation localization has become more challenging. Existing methods often struggle with limited semantic understanding and insufficient spatial detail capture, especially in complex scenarios. To address these issues, we propose a novel multimodal text-guided framework for image manipulation localization. By fusing textual prompts with image features, our approach enhances the model’s ability to identify manipulated regions. We introduce a Multimodal Interaction Prompt Module (MIPM) that uses cross-modal attention mechanisms to align visual and textual information. Guided by multimodal prompts, our Vision Transformer-based model accurately localizes forged areas in images. Extensive experiments on public datasets, including CASIAv1 and Columbia, show that our method outperforms existing approaches. Specifically, on the CASIAv1 dataset, our approach achieves an F1 score of 0.734, surpassing the second-best method by 1.3%. These results demonstrate the effectiveness of our multimodal fusion strategy. The code is available at <span><span>https://github.com/Makabaka613/MPG-ViT</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":54755,"journal":{"name":"Journal of Visual Communication and Image Representation","volume":"116 ","pages":"Article 104736"},"PeriodicalIF":3.1,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146079535","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Enhancing temporal action localization through cross-modal and cross-structural knowledge distillation","authors":"Yue Yu,&nbsp;Cheng Wang,&nbsp;Yuxin Shi","doi":"10.1016/j.jvcir.2026.104734","DOIUrl":"10.1016/j.jvcir.2026.104734","url":null,"abstract":"<div><div>This paper proposes Cross-Modal and Cross-Structure distillation for rgb-based temporal action detection(C2MS-Net), a novel fully supervised approach for enhancing temporal action localization by leveraging cross-modal and cross-structural distillation techniques. By integrating information from multiple modalities and structural representations, C2MS-Net significantly improves the discriminative power of action proposals. A distillation framework is introduced, which transfers knowledge from a teacher model trained on rich multi-modal data to a more efficient student model. This approach not only enhances temporal localization accuracy but also improves the robustness of action detection against visual content variations. Extensive experiments on benchmark datasets demonstrate that the proposed C2MS-Net performs competitively with or surpasses state-of-the-art methods, particularly at lower and mid-range tIoU thresholds, while offering substantial gains in computational efficiency. By eliminating the need for optical flow extraction, the proposed method substantially reduces computational complexity, achieving faster inference speeds and smaller model sizes without compromising accuracy. Code, dataset and models are available at: <span><span>https://github.com/wangcheng666/ActionFormer</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":54755,"journal":{"name":"Journal of Visual Communication and Image Representation","volume":"116 ","pages":"Article 104734"},"PeriodicalIF":3.1,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146174126","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Progressively multi-scale feature fusion for semantic segmentation","authors":"Guoqing Zhang ,&nbsp;Shichao Kan ,&nbsp;Yigang Cen ,&nbsp;Yi Cen ,&nbsp;Qi Cao ,&nbsp;Yansen Huang ,&nbsp;Ming Zeng","doi":"10.1016/j.jvcir.2026.104739","DOIUrl":"10.1016/j.jvcir.2026.104739","url":null,"abstract":"<div><div>A fundamental challenge in semantic segmentation is the discriminative learning of pixel-level features. Various semantic segmentation methods and decoders in the literature have been reported to address this challenge. These methods involve directly upsampling feature maps of different sizes and then concatenating them along the channel dimension to generate pixel-level features. However, direct upsampling of feature maps can result in the misalignment of information at the pixel level, leading to suboptimal performance. In this paper, we introduce a novel solution called the Progressive Multi-Scale Feature Fusion (PMSFF) decoder to overcome this issue. Specifically, we develop a lightweight feed-forward network and atrous convolution layer, that are combined as a fusion module to fuse feature maps from adjacent layers. This fusion module is applied to different segments of a network to aggregate all feature maps for semantic segmentation. The fusion module is characterized by a simple and convenient structure with fewer parameters, which can be flexibly embedded into both Convolutional Neural Networks (CNNs) and Transformers to achieve progressive multi-scale pixel-level feature fusion. Extensive experiments on benchmark datasets have been conducted. The results illustrate the effectiveness and efficiency of the proposed module.</div></div>","PeriodicalId":54755,"journal":{"name":"Journal of Visual Communication and Image Representation","volume":"116 ","pages":"Article 104739"},"PeriodicalIF":3.1,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146174130","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"MGLA-DSNet: Multi-head global-local attention-enabled dual-stream network for weakly supervised video anomaly detection","authors":"Rashmiranjan Nayak,&nbsp;Umesh Chandra Pati,&nbsp;Santos Kumar Das","doi":"10.1016/j.jvcir.2026.104744","DOIUrl":"10.1016/j.jvcir.2026.104744","url":null,"abstract":"<div><div>Video Anomaly Detection (VAD) is the process of identifying anomalous events by analyzing spatiotemporal patterns in video. Furthermore, VAD is a complex task due to difficulties in obtaining frame-level annotations, data imbalance issues, and the equivocal and context-dependent nature of video anomalies. To address these issues, this article presents a weakly supervised learning-based Multi-head Global-Local Attention-enabled Dual-Stream Network (MGLA-DSNet) that effectively utilizes spatial (appearance) and temporal (motion) features, with an emphasis on context dependency. The proposed model uses two streams to extract RGB and optical flow features corresponding to appearance (spatial) and motion (temporal) properties, respectively. Subsequently, multi-head global and location attention with adaptive gating and head-wise specialization is applied to the concatenated RGB and Flow features to efficiently model global and local contexts, respectively, using multiple instance learning Finally, the proposed MGLA-DSNet model outperforms state-of-the-art methods across three benchmark datasets, including CUHK Avenue, ShanghaiTech Campus, and UCF-Crime.</div></div>","PeriodicalId":54755,"journal":{"name":"Journal of Visual Communication and Image Representation","volume":"116 ","pages":"Article 104744"},"PeriodicalIF":3.1,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146174129","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"CKCR: Context-aware knowledge construction and retrieval for knowledge-based visual question answering","authors":"Fengjuan Wang ,&nbsp;Jiayi Liu ,&nbsp;Ruonan Zhang,&nbsp;Zhengxue Li,&nbsp;Feng Zhang,&nbsp;Gaoyun An","doi":"10.1016/j.jvcir.2026.104711","DOIUrl":"10.1016/j.jvcir.2026.104711","url":null,"abstract":"<div><div>Knowledge-based Visual Question Answering (KB-VQA) requires models to integrate visual content with external knowledge to answer questions, which is crucial for building intelligent systems capable of real-world understanding. However, effectively incorporating external knowledge into visual reasoning faces three major challenges: the incompleteness of external knowledge bases leads to missing knowledge for many specific visual scenarios, semantic gaps exist between retrieved textual knowledge and visual content making alignment difficult, and effective mechanisms for fusing heterogeneous knowledge sources are lacking. While Multimodal Large Language Models(MLLMs) have demonstrated strong performance in visual understanding tasks, but face notable challenges in KB-VQA, particularly in knowledge utilization efficiency and semantic alignment, which seriously limits the reasoning depth and robustness. To address these problems, a Context-aware Knowledge Construction and Retrieval (CKCR) method is proposed for knowledge-based VQA, which includes the following three modules. The multi-granularity knowledge retrieval module constructs joint query vector based on the multi-dimensional embedding representation of images and questions, accurately obtaining explicit knowledge that is highly matched with the context. The vision-to-knowledge generation module supplements fine-grained semantic clues from the perspective of visual content, generating visual knowledge closely related to the image and making up for the expression limitations of general knowledge. To achieve deep alignment of knowledge representation, the knowledge adaptive learning module accurately embeds multi-source knowledge into the semantic space of MLLM by introducing a learnable knowledge mapping mechanism. Experimental evaluation on OK-VQA and A-OKVQA dataset shows the CKCR outperforms state-of-the-art methods of the same-scale. Ablation experiments and visualization analysis demonstrate the superiority of CKCR in its perception of fine-grained visual information and its ability to align knowledge semantics. Our code will be released on GitHub: <span><span>https://github.com/fjwang3/CKCR</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":54755,"journal":{"name":"Journal of Visual Communication and Image Representation","volume":"116 ","pages":"Article 104711"},"PeriodicalIF":3.1,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145981617","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"eGoRG: GPU-accelerated depth estimation for immersive video applications based on graph cuts","authors":"Jaime Sancho ,&nbsp;Manuel Villa ,&nbsp;Miguel Chavarrias ,&nbsp;Rubén Salvador ,&nbsp;Eduardo Juarez ,&nbsp;César Sanz","doi":"10.1016/j.jvcir.2026.104727","DOIUrl":"10.1016/j.jvcir.2026.104727","url":null,"abstract":"<div><div>Immersive video is gaining relevance across various fields, but its integration into real applications remains limited due to the technical challenges of depth estimation. Generating accurate depth maps is essential for 3D rendering, yet high-quality algorithms can require hundreds of seconds to produce a single frame. While real-time depth estimation solutions exist — particularly monocular deep learning-based methods and active sensors such as time-of-flight or plenoptic cameras — their depth accuracy and multiview consistency are often insufficient for depth image-based rendering (DIBR) and immersive video applications. This highlights the persistent challenge of jointly achieving real-time performance and high-quality, correlated depth across views. This paper introduces eGoRG, a GPU-accelerated depth estimation algorithm based on MPEG DERS, which employs graph cuts to achieve high-quality results. eGoRG contributes a novel GPU-based graph cuts stage, integrating block-based push-relabel acceleration and a simplified alpha expansion method. These optimizations deliver quality comparable to leading graph-cut approaches while greatly improving speed. Evaluation on an MPEG multiview dataset and a static NeRF dataset demonstrates the algorithm’s effectiveness across different scenarios.</div></div>","PeriodicalId":54755,"journal":{"name":"Journal of Visual Communication and Image Representation","volume":"116 ","pages":"Article 104727"},"PeriodicalIF":3.1,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146024843","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Image copy-move forgery detection using three-stage matching with constraints","authors":"Panpan Niu,&nbsp;Hongxin Wang,&nbsp;Xingqi Wang","doi":"10.1016/j.jvcir.2026.104709","DOIUrl":"10.1016/j.jvcir.2026.104709","url":null,"abstract":"<div><div>Copy-move forgery is one of the most commonly used manipulations for tampering digital images. In recent years, keypoint-based detection methods have achieved encouraging results, but there are still several shortcomings that can be improved. First, unability to generate sufficient keypoints in small or smooth regions, causing detection failure. Second, lack of robust and discriminative descriptors for image keypoints, resulting in false matches. Third, high computational cost of image keypoints matching. To tackle this challenge, we present a new keypoint-based image copy-move forgery detection (CMFD) using three-stage matching with constraints. In keypoint extraction, we extract sufficient SIFT keypoints by adaptively enlarging image and enhancing image contrast. In feature description, we adopt the combination of complex and real values of Polar Harmonic Fourier Moments (PHFMs) as the PHFMs-based hybrid feature vector of each keypoint, which substantially enhances the differentiation of the features. In feature matching, we present a fast stratification approach based on SLIC and locally optimal orientation pattern (LOOP), and utilize the stratification results as the constraints of matching, which can reduce the search space. Then a high-precision three-stage matching strategy based on amplitude information, phase information and distance information is executed. In post-processing, the location of the tampered regions is finally determined by one-step filtering and one-step clustering. Extensive experimental results show the superiority of the proposed method over the existing representative CMFD techniques.</div></div>","PeriodicalId":54755,"journal":{"name":"Journal of Visual Communication and Image Representation","volume":"116 ","pages":"Article 104709"},"PeriodicalIF":3.1,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145950136","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"NCC-FDM: Frequency-domain diffusion model driven by non-physical-domain color correction for underwater image enhancement","authors":"Guanglin Qu,&nbsp;Xiuman Liang,&nbsp;Zhendong Liu,&nbsp;Haifeng Yu","doi":"10.1016/j.jvcir.2026.104740","DOIUrl":"10.1016/j.jvcir.2026.104740","url":null,"abstract":"<div><div>Underwater environments have the characteristics of light absorption and scattering. Images captured in such environments commonly suffer from multiple degradation issues such as color bias, haze, detail loss and low contrast, which further severely interfere with downstream underwater vision tasks. To addressing the challenges of underwater image enhancement, we propose a non-physical color correction driven frequency domain diffusion model (NCC-FDM). The model combines non-physical color correction with a conditional diffusion model. Firstly, we design a non-physical color correction stage (NCCS) to rapidly address severe color shifts in underwater datasets. Image color deviations are corrected through the combined application of rapid optical wave compensation and the gray scale world method. Secondly, we consider the different degrees of degradation between high frequency images and low-frequency images in underwater images. We design a frequency domain conditional diffusion model based on discrete wavelet transform to process the low-frequency components of color-corrected images. A hybrid high-frequency enhancement module (HHEM) is proposed to restore detail and structural information in our images. The module separately enhances the high-frequency components of different spatial dimensions. The enhancement is based on the principle that the proportion of information and noise contained in the high-frequency components varies across different spatial dimensions. Finally, we design a joint loss function to optimize the frequency domain diffusion model. The joint loss function includes noise loss, reconstruction loss and high-frequency loss. Comprehensive evaluation across four public underwater datasets demonstrates that the proposed NCC-FDM algorithm achieves outstanding performance in both visual quality and evaluation metrics.</div></div>","PeriodicalId":54755,"journal":{"name":"Journal of Visual Communication and Image Representation","volume":"116 ","pages":"Article 104740"},"PeriodicalIF":3.1,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146174128","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Global–local dual-branch network with local feature enhancement for visual tracking","authors":"Yuanyun Wang,&nbsp;Lingtao Zhou,&nbsp;Zhuo An,&nbsp;Lei Sun,&nbsp;Min Hu,&nbsp;Jun Wang","doi":"10.1016/j.jvcir.2026.104725","DOIUrl":"10.1016/j.jvcir.2026.104725","url":null,"abstract":"<div><div>Vision Transformers (ViT) have been widely applied due to their excellent performance. Compared with CNN models, ViT models are more difficult to train and require more training samples because they cannot effectively utilize high-frequency local information. In this paper we propose an efficient tracking framework based on global and local feature extraction, and an enhancement module. To address the high-frequency local information neglected by general ViT-based trackers, we design an effective local branch architecture to capture the information. For local feature extraction and enhancement, we design a local branch, which aggregates local information by using shared weights; it utilizes the optimized context-aware weights to enhance the local features. The integration of the attention mechanism in the global and local branches enables the tracker to perceive both high-frequency local information and low-frequency global information simultaneously. Experimental comparisons show that the tracker achieves superior results and proves the generalization ability and effectiveness. Code will be available at <span><span>https://github.com/WangJun-CV/GLDTrack</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":54755,"journal":{"name":"Journal of Visual Communication and Image Representation","volume":"116 ","pages":"Article 104725"},"PeriodicalIF":3.1,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146079536","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}