IEEE Transactions on Emerging Topics in Computational Intelligence最新文献_第10页

DualC: Drug-Drug Interaction Prediction Based on Dual Latent Feature Extractions 基于双重潜在特征提取的药物-药物相互作用预测

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2024-12-02 DOI: 10.1109/TETCI.2024.3502414

Lin Guo;Xiujuan Lei;Lian Liu;Ming Chen;Yi Pan

{"title":"DualC: Drug-Drug Interaction Prediction Based on Dual Latent Feature Extractions","authors":"Lin Guo;Xiujuan Lei;Lian Liu;Ming Chen;Yi Pan","doi":"10.1109/TETCI.2024.3502414","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3502414","url":null,"abstract":"Drug-Drug Interaction is characterized by a modification in the action of one drug due to its concurrent use with another. It involves the safety and universality of drugs, and is one of the most meaningful issues in clinical drug combination therapy and drug development. We prefer to use computational methods to achieve DDI prediction in order to achieve large-scale prediction. This article designs a DDI prediction model DualC based on the layer attention mechanism of Graph Convolutional Network and 1 Dimensional-Convolutional Neural Network to extract topological and structural information of drugs. First, the DDI network is obtained from the drug relationship data in the database and the drug similarity network is calculated with the help of drug target features, then they are constructed into a heterogeneous network. Next, the layer attention mechanism and Graph Convolutional Network are used to learn the topological information. Subsequently, the structural information is acquired from the chemical substructure similarity matrix utilizing 1 Dimensional-Convolutional Neural Network. Finally, use the sigmoid function for DDI prediction. The experimental results show advantages of DualC which AUC reaches 0.965 and ACC reaches 0.973. The case study further proves DualC has certain practical significance.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 1","pages":"946-960"},"PeriodicalIF":5.3,"publicationDate":"2024-12-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143106794","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Image Manipulation Localization Using Dual-Shallow Feature Pyramid Fusion and Boundary Contextual Incoherence Enhancement 基于双浅特征金字塔融合和边界上下文不相干增强的图像处理定位

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2024-11-29 DOI: 10.1109/TETCI.2024.3500025

Yan Xiang;Xiaochen Yuan;Kaiqi Zhao;Tong Liu;Zhiyao Xie;Guoheng Huang;Jianqing Li

{"title":"Image Manipulation Localization Using Dual-Shallow Feature Pyramid Fusion and Boundary Contextual Incoherence Enhancement","authors":"Yan Xiang;Xiaochen Yuan;Kaiqi Zhao;Tong Liu;Zhiyao Xie;Guoheng Huang;Jianqing Li","doi":"10.1109/TETCI.2024.3500025","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3500025","url":null,"abstract":"This paper proposes a novel end-to-end network for Image Manipulation Localization (IML) comprising three modules: feature fusion, encoder, and decoder. To address the limitations of current DNN-based IML algorithms in accessing global features and segmenting tampered edges, we propose a Dual-shallow Feature Pyramid Fusion (DFPF) module. The DFPF module integrates semantic and texture features through a bidirectional pathway, forming RGB Feature Pyramids (RGBFP) and Local Textual Feature Pyramids (LTFP) using dual Hybrid ResNet50s in a ’Siamese' configuration. These feature pyramids are merged via multi-scale fusion to enhance global pyramid features for decoding. The LTFP branch includes a Pre-processing Block, Parallel Multi-Scale Convolution (PMSC), or Channel Split High-frequency Convolution (CSHC) to capture local textual features and subtle manipulation traces. The Encoder employs Transformer layers for robust global representations. At the same time, the Decoder uses Cascaded Boundary Context Inconsistent Enhancement (BCIE) Blocks to reconstruct a coarse-to-fine binary mask, enhancing texture inconsistencies at manipulated region boundaries. Additionally, we introduce an automated method for generating a large-scale forgery dataset via Photoshop Scripting, reducing labor costs. Our model effectively locates tampered regions of various shapes and sizes, improving boundary anomaly detection. Extensive experimental results demonstrate that our method significantly outperforms existing state-of-the-art models.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 4","pages":"2858-2868"},"PeriodicalIF":5.3,"publicationDate":"2024-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144687699","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Two-Stage Privacy-Preserving Domain Adaptation for Industrial Time-Series Prediction 面向工业时间序列预测的两阶段隐私保护域自适应

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2024-11-29 DOI: 10.1109/TETCI.2024.3502418

Zidi Jia;Lei Ren;Haiteng Wang;Yuanjun Laili

{"title":"A Two-Stage Privacy-Preserving Domain Adaptation for Industrial Time-Series Prediction","authors":"Zidi Jia;Lei Ren;Haiteng Wang;Yuanjun Laili","doi":"10.1109/TETCI.2024.3502418","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3502418","url":null,"abstract":"Domain adaptation (DA), as an emerging computational intelligence technology, is crucial for industrial time-series prediction, since various operating environments and tasks of industrial equipment will lead to variants in the distribution of monitoring data. Recently, many DA methods have been proposed to adapt to cross-domain industrial scenarios with data distribution shifting. However, due to privacy preserving concerns, data owners are reluctant to share their data, resulting in inaccessible source data. The artificial intelligence (AI) model trained by the inaccessible source data can be only applied as a blackbox. This makes it difficult to transfer source domain knowledge to the target domain. To solve this issue, we propose a two-stage source-free domain adaptation method for unsupervised knowledge transfer for industrial time-series prediction. At the first stage, an adversarial training method is proposed to improve the model's ability to represent data in the target domain, which may significantly differ from the source distribution. At the second stage, an unsupervised feature alignment method based on mean-teacher is proposed to align the target domain data with source knowledge. Additionally, we defined two contrastive loss functions to strengthen the consistent representation of target data. Experiments conducted on datasets N-CMAPSS and FEMOTO-ST demonstrate the effectiveness of the proposed method.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 4","pages":"2846-2857"},"PeriodicalIF":5.3,"publicationDate":"2024-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144687665","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Privacy-Enhanced Federated GNN Inference Against Adversarial Example Attack 针对对抗性示例攻击的隐私增强联邦GNN推理

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2024-11-28 DOI: 10.1109/TETCI.2024.3502434

Guanghui He;Yanli Ren;Jingyuan Jiang;Guorui Feng;Xinpeng Zhang

{"title":"Privacy-Enhanced Federated GNN Inference Against Adversarial Example Attack","authors":"Guanghui He;Yanli Ren;Jingyuan Jiang;Guorui Feng;Xinpeng Zhang","doi":"10.1109/TETCI.2024.3502434","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3502434","url":null,"abstract":"Graph neural networks (GNNs) have become a powerful tool for processing and learning graph data. However, due to the existence of data silos, the privacy of data and the processing result is an important concern. Meanwhile, the malicious example will result in the incorrect output of the model. For the above concerns, this paper proposes privacy-enhanced federated graph network inference against adversarial example attack. Specifically, we adopt secret sharing and homomorphic encryption to ensure the privacy of graph data, where the user can get the final inference, and the server holds nothing except the model parameters. Moreover, in order to prevent malicious users from interfering with the accuracy of the model, an adversarial example detection mechanism on the ciphertext is designed to identify local embedding submitted by malicious users. During the whole process, both local and global embedding are both protected. The experimental results show that the model accuracy is about 69% and 66% with malicious samples on Cora and Citeseer in the domain of ciphertext respectively and they are nearly same as 70% and 69% in the domain of plaintext, which shows the effectiveness of our protocol.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 4","pages":"2818-2829"},"PeriodicalIF":5.3,"publicationDate":"2024-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144687736","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

PEAC-HNF: A Novel Multi-Objective Evolutionary Algorithm for Split Delivery Vehicle Routing With Three-Dimensional Loading Constraints 基于多目标进化算法的三维载荷约束下分步配送车辆路径优化

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2024-11-28 DOI: 10.1109/TETCI.2024.3499992

Han Zhang;Qing Li;Xin Yao

{"title":"PEAC-HNF: A Novel Multi-Objective Evolutionary Algorithm for Split Delivery Vehicle Routing With Three-Dimensional Loading Constraints","authors":"Han Zhang;Qing Li;Xin Yao","doi":"10.1109/TETCI.2024.3499992","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3499992","url":null,"abstract":"The Split Delivery Vehicle Routing Problem with Three-Dimensional Loading Constraints (3L-SDVRP) integrates routing and packing problems, aiming to maximize the vehicle load efficiency and minimize the total travel distance. Solving 3L-SDVRP is critical for logistics and transportation industries. However, achieving an appropriate balance between exploration (searching for new solutions) and exploitation (refining known solutions) in metaheuristic algorithms for 3L-SDVRP, especially under limited computational resources, remains challenging. Furthermore, the application of multi-objective optimization algorithms to the 3L-SDVRP remains a largely unexplored area, particularly when considering the inherent trade-offs between the two conflicting objectives. To address these challenges, this paper introduces a new Pareto-based Evolutionary Algorithm with Concurrent crossover and Hierarchical Neighborhood Filtering mutation (PEAC-HNF), distinguished by its novel Hierarchical Neighborhood Filtering (HNF) mutation. The HNF mutation uses diverse neighborhood structures to generate offspring, adopts a hierarchical strategy prioritizing individuals with higher nondomination ranks, and incorporates an offspring filtering process to save computational resources. HNF allows PEAC-HNF to improve its exploitation capabilities while maintaining exploration strengths, achieving a balanced performance. Comparisons with state-of-the-art algorithms across various problem instances (242 instances in total) demonstrate the effectiveness of PEAC-HNF. Further analysis highlights the critical role of the HNF mutation in enhancing algorithmic performance. The utilization of the HNF mutation can extend beyond PEAC-HNF to other complex optimization problems.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 4","pages":"2830-2845"},"PeriodicalIF":5.3,"publicationDate":"2024-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144687761","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Impact of Strategic Sampling and Supervision Policies on Semi-Supervised Learning 策略抽样和监督策略对半监督学习的影响

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2024-11-27 DOI: 10.1109/TETCI.2024.3502453

Shuvendu Roy;Ali Etemad

{"title":"Impact of Strategic Sampling and Supervision Policies on Semi-Supervised Learning","authors":"Shuvendu Roy;Ali Etemad","doi":"10.1109/TETCI.2024.3502453","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3502453","url":null,"abstract":"In semi-supervised representation learning frameworks, when the number of labelled data is very scarce, the quality and representativeness of these samples become increasingly important. Existing literature on semi-supervised learning randomly sample a limited number of data points for labelling. All these labelled samples are then used along with the unlabelled data throughout the training process. In this work, we ask two important questions in this context: 1) does it matter which samples are selected for labelling? 2) does it matter how the labelled samples are used throughout the training process along with the unlabelled data? To answer the first question, we explore a number of unsupervised methods for selecting specific subsets of data to label (without prior knowledge of their labels), with the goal of maximizing representativeness w.r.t. the unlabelled set. Then, for our second line of inquiry, we define a variety of different label injection strategies in the training process. Extensive experiments on four popular datasets, CIFAR-10, CIFAR-100, SVHN, and STL-10, show that unsupervised selection of samples that are more representative of the entire data improves performance by up to <inline-formula><tex-math>$sim$</tex-math></inline-formula>2% over the existing semi-supervised frameworks such as MixMatch, ReMixMatch, FixMatch and others with random sample labelling. We show that this boost could even increase to 7.5% for very few-labelled scenarios. However, our study shows that gradually injecting the labels throughout the training procedure does not impact the performance considerably versus when all the existing labels are used throughout the entire training.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 4","pages":"2806-2817"},"PeriodicalIF":5.3,"publicationDate":"2024-11-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144687694","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

LoCATe-GAT: Modeling Multi-Scale Local Context and Action Relationships for Zero-Shot Action Recognition LoCATe-GAT：零射击动作识别的多尺度局部环境和动作关系建模

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2024-11-27 DOI: 10.1109/TETCI.2024.3499995

Sandipan Sarma;Divyam Singal;Arijit Sur

{"title":"LoCATe-GAT: Modeling Multi-Scale Local Context and Action Relationships for Zero-Shot Action Recognition","authors":"Sandipan Sarma;Divyam Singal;Arijit Sur","doi":"10.1109/TETCI.2024.3499995","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3499995","url":null,"abstract":"The increasing number of actions in the real world makes it difficult for traditional deep-learning models to recognize unseen actions. Recently, pretrained contrastive image-based visual-language (I-VL) models have been adapted for efficient “zero-shot” scene understanding. Pairing such models with transformers to implement temporal modeling has been rewarding for zero-shot action recognition (ZSAR). However, the significance of modeling the local spatial context of objects and action environments remains unexplored. In this work, we propose a ZSAR framework called <italic>LoCATe-GAT</i>, comprising a novel Local Context-Aggregating Temporal transformer (LoCATe) and a Graph Attention Network (GAT). Specifically, image and text encodings extracted from a pretrained I-VL model are used as inputs for LoCATe-GAT. Motivated by the observation that object-centric and environmental contexts drive both distinguishability and functional similarity between actions, LoCATe captures multi-scale local context using dilated convolutional layers during temporal modeling. Furthermore, the proposed GAT models semantic relationships between classes and achieves a strong synergy with the video embeddings produced by LoCATe. Extensive experiments on four widely-used benchmarks – UCF101, HMDB51, ActivityNet, and Kinetics – show we achieve state-of-the-art results. Specifically, we obtain relative gains of 3.8% and 4.8% on these datasets in conventional and 16.6% on UCF101in generalized ZSAR settings. For large-scale datasets like ActivityNet and Kinetics, our method achieves a relative gain of 31.8% and 27.9%, respectively, over the previous methods. Additionally, we gain 25.3% and 18.4% on UCF101 and HMDB51 as per the recent “TruZe” evaluation protocol.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 4","pages":"2793-2805"},"PeriodicalIF":5.3,"publicationDate":"2024-11-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144687691","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Large-Scale Hierarchical Medical Image Retrieval Based on a Multilevel Convolutional Neural Network 基于多层卷积神经网络的大规模分层医学图像检索

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2024-11-27 DOI: 10.1109/TETCI.2024.3502404

Chung-Ming Lo;Cheng-Yeh Hsieh

{"title":"Large-Scale Hierarchical Medical Image Retrieval Based on a Multilevel Convolutional Neural Network","authors":"Chung-Ming Lo;Cheng-Yeh Hsieh","doi":"10.1109/TETCI.2024.3502404","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3502404","url":null,"abstract":"Presently, with advancements in medical imaging modalities, various imaging methods are widely used in clinics. To efficiently assess and manage the images, in this paper, a content-based medical image retrieval (CBMIR) system is suggested as a clinical tool. A global medical image database is established through a collection of data from more than ten countries and dozens of sources, schools and laboratories. The database has more than 536 294 medical images, including 14 imaging modalities, 40 organs and 52 diseases. A multilevel convolutional neural network (MLCNN) using hierarchical progressive feature learning is subsequently proposed to perform hierarchical medical image retrieval, including multiple levels of image modalities, organs and diseases. At each classification level, a dense block is trained through a labeled classification. With the epochs increasing, four training stages are performed to simultaneously train the three levels with different weights of the loss function. Then, the trained features are used in the CBMIR system. The results show that using the MLCNN on a representative dataset can achieve a mAP of 0.86, which is higher than the 0.71 achieved by ResNet152 in the literature. Applying the hierarchical progressive feature learning can achieve a 12%-16% performance improvement in CNNs and outperform vision Transformer with only 63% of the training time. The proposed representative image selection and multilevel architecture improves the efficiency and precision of retrieving large-scale medical image databases.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 4","pages":"2782-2792"},"PeriodicalIF":5.3,"publicationDate":"2024-11-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144687675","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Adaptive Constrained IVAMGGMM: Application to Mental Disorders Detection 自适应约束IVAMGGMM在精神障碍检测中的应用

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2024-11-27 DOI: 10.1109/TETCI.2024.3500023

Ali Algumaei;Muhammad Azam;Nizar Bouguila

{"title":"Adaptive Constrained IVAMGGMM: Application to Mental Disorders Detection","authors":"Ali Algumaei;Muhammad Azam;Nizar Bouguila","doi":"10.1109/TETCI.2024.3500023","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3500023","url":null,"abstract":"The demand for adaptable approaches to analyze extensive fMRI data is growing, focusing on capturing population patterns while preserving individual uniqueness. Independent component analysis (ICA) is increasingly used to uncover spatio-temporal patterns in brain imaging but struggles with separating correlated sources in multivariate data like fMRI. For that, we propose an ICA-based multivariate generalized Gaussian mixture model combined with the constrained ICA to form the cICA-MGGMM. This model relaxes the independence assumption of ICA. Also, we propose the adaptive constrained ICA-MGGMM (acICA-MGGMM) to adaptively control the association between reference signals and estimated sources. Independent vector analysis (IVA) calculates global spatial and temporal patterns from multi-subject fMRI data while preserving individual variability but performs poorly with large datasets and weak component correlations. This paper proposes integrating reference signals into the formulation to address the problem and provide guidance in high-dimensional situations. For that, we propose cIVA-MGGMM to address ICA limitations for multivariate data, offering a framework for references but relying on user-defined constraint parameters to enforce reference-estimated sources associations. To tackle these limitations, we introduce the adaptive cIVA-MGGMM (acIVA-MGGMM) to adapt and separate the activated brain sources. This model employs a full covariance matrix, which consider the feature correlation. Our four constrained methods incorporate prior information about the sources into the ICA and IVA models to address the limitations of ICA and IVA in high-dimensional data. We validate our models on simulation, Alzheimer's, Schizophrenia, EEG, and ADHD datasets, demonstrating superior performance over base models.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 3","pages":"2506-2530"},"PeriodicalIF":5.3,"publicationDate":"2024-11-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144148148","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Triplet Bridge for Zero-Shot Sketch-Based Image Retrieval 基于零镜头草图的图像检索的三重桥

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2024-11-26 DOI: 10.1109/TETCI.2024.3502430

Jiahao Zheng;Yu Tang;Dapeng Wu

{"title":"Triplet Bridge for Zero-Shot Sketch-Based Image Retrieval","authors":"Jiahao Zheng;Yu Tang;Dapeng Wu","doi":"10.1109/TETCI.2024.3502430","DOIUrl":"https://doi.org/10.1109/TETCI.2024.3502430","url":null,"abstract":"Zero-Shot Sketch-Based Image Retrieval (ZS-SBIR) has always been a hard nut to crack due to the scarcity of sketch data and the abstract visual information contained in sketches. Previous works focus on designing various network architectures and using the gold standard triplet loss to solve ZS-SBIR, but they have always encountered obstacles in enhancing model generalization and extracting abstract visual information. In contrast, this work proposes a concise and effective Triplet Bridge (TriBri) framework to clear these obstacles fundamentally. Specifically, we use InfoNCE as the core to construct cross-modal representations between images and sketches, which can increase the margin between feature clusters with different categories in the representation space and improve the generalization of the model. Furthermore, we introduce text with abstract properties into the framework to construct a ternary relationship, and the three heterogeneous gaps between text, image, and sketch modalities are connected by InfoNCE. In this process, the common abstract visual cues in both images and sketches can be captured by the feature extractor with the guiding of text abstract information. Ultimately, comprehensive experiments on three commonly used datasets (i.e., TU-Berlin, Sketchy, and QuickDraw) validate that our framework can effectively solve these obstacles in a simple yet powerful manner. Furthermore, compared to state-of-the-art methods, the proposed TriBri exhibits comprehensive performance superiority.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 2","pages":"2014-2025"},"PeriodicalIF":5.3,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143706553","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0