IEEE Transactions on Emerging Topics in Computational Intelligence最新文献_第4页

Exploiting High Performance Spiking Neural Networks With Efficient Spiking Patterns 利用高效脉冲模式开发高性能脉冲神经网络

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2025-02-21 DOI: 10.1109/TETCI.2025.3540408

Guobin Shen;Dongcheng Zhao;Yi Zeng

{"title":"Exploiting High Performance Spiking Neural Networks With Efficient Spiking Patterns","authors":"Guobin Shen;Dongcheng Zhao;Yi Zeng","doi":"10.1109/TETCI.2025.3540408","DOIUrl":"https://doi.org/10.1109/TETCI.2025.3540408","url":null,"abstract":"Spiking Neural Networks (SNNs) use discrete spike sequences to transmit information, which significantly mimics the information transmission of the brain. Although this binarized form of representation dramatically enhances the energy efficiency and robustness of SNNs, it also leaves a large gap between the performance of SNNs and Artificial Neural Networks based on real values. There are many different spike patterns in the brain, and the dynamic synergy of these spike patterns greatly enriches the representation capability. Inspired by spike patterns in biological neurons, this paper introduces the dynamic Burst pattern and designs the Leaky Integrate and Fire or Burst (IF&B) neuron that can make a trade-off between short-time performance and dynamic temporal performance from the perspective of network information capacity. IF&B neuron exhibits three modes, resting, Regular spike, and Burst spike. The burst density of the neuron can be adaptively adjusted, which significantly enriches the characterization capability. We also propose a decoupling method that can losslessly decouple IF&B neurons into equivalent LIF neurons, which demonstrates that IF&B neurons can be efficiently implemented on neuromorphic hardware. We conducted experiments on the static datasets CIFAR10, CIFAR100, and ImageNet, which showed that we greatly improved the performance of the SNNs while significantly reducing the network latency. We also conducted experiments on neuromorphic datasets DVS-CIFAR10 and NCALTECH101 and showed that we achieved state-of-the-art with a small network structure.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 2","pages":"1480-1489"},"PeriodicalIF":5.3,"publicationDate":"2025-02-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143706629","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MssNet: An Efficient Spatial Attention Model for Early Recognition of Alzheimer's Disease MssNet：早期识别阿尔茨海默病的有效空间注意模型

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2025-02-19 DOI: 10.1109/TETCI.2025.3537942

Jiayu Ye;Dan Pan;An Zeng;Yiqun Zhang;Qiuping Chen;Yang Liu

{"title":"MssNet: An Efficient Spatial Attention Model for Early Recognition of Alzheimer's Disease","authors":"Jiayu Ye;Dan Pan;An Zeng;Yiqun Zhang;Qiuping Chen;Yang Liu","doi":"10.1109/TETCI.2025.3537942","DOIUrl":"https://doi.org/10.1109/TETCI.2025.3537942","url":null,"abstract":"Deep learning models are widely used in medical image-guided disease recognition and have achieved outstanding performance. Voxel-based models are typically the default choice for deep learning-based MRI analysis, which require high computational resources and large data volumes, making them inefficient for rapid disease screening. Simultaneously, the existing Alzheimer's disease (AD) recognition model is primarily comprised of Convolutional Neural Network (CNN) structures. With the increasing of the network depth, the fine-grained details of global features tend to be partially lost. Therefore, we propose a Multi-scale spatial self-attention Network (MssNet) that effectively captures both coarse-grained and fine-grained features. We design to select the target slice based on image entropy to achieve efficient slice-based AD recognition. To capture multi-level spatial information, a novel spatial attention mechanism and spatial self-attention attention are designed. The former is utilized to collect critical spatial information and identify areas that are likely to be lesions, the latter investigates the relationship between features in different image regions through spatial interaction by pure convolutional blocks. MssNet fully utilizes multi-scale information at different granularities for spatial feature interaction, providing it with strong modeling and information understanding capabilities. It has achieved excellent performance in the recognition tasks of Alzheimer's Disease Neuroimaging Initiative (ADNI) and Open Access Series of Imaging Studies (OASIS) datasets. Moreover, MssNet is a lightweight model involving lower scale parameters against the Voxel-based ones, while demonstrating strong generalization capability.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 2","pages":"1454-1468"},"PeriodicalIF":5.3,"publicationDate":"2025-02-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143706699","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Swarm Optimization With Intra- and Inter-Hierarchical Competition for Large-Scale Berth Allocation and Crane Assignment 大规模泊位分配与起重机分配的群内与群间竞争优化

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2025-02-19 DOI: 10.1109/TETCI.2025.3529876

Yanghe Zou;Peilan Xu;Hao Dai;Heng Song;Wenjian Luo

{"title":"Swarm Optimization With Intra- and Inter-Hierarchical Competition for Large-Scale Berth Allocation and Crane Assignment","authors":"Yanghe Zou;Peilan Xu;Hao Dai;Heng Song;Wenjian Luo","doi":"10.1109/TETCI.2025.3529876","DOIUrl":"https://doi.org/10.1109/TETCI.2025.3529876","url":null,"abstract":"The trend of global economic integration has fostered the prosperity of the maritime transportation industry, which has placed higher demands on the construction of automated container terminals, and the optimization of the integrated berth allocation and crane assignment problems (BACAPs) is a key link. Currently, population-based computational intelligence methods have attracted attention on BACAPs, but on small-scale cases and simplified problem models. In this paper, we propose a novel swarm optimization with intra- and inter-hierarchical competition (I2HCSO) for addressing large-scale BACAPs, which is a major challenge in container terminals. First, we construct a hierarchical model with better particles at the higher hierarchy, and the populations at different hierarchies are divided into several sub-swarm. Then, we design an intra- and inter-hierarchical competitive mechanism to balance the exploration and exploitation of the population, in which intra-hierarchical competition is carried out within sub-swarm at any hierarchy, whereas inter-hierarchical competition occurs in different sub-swarms of neighboring hierarchies. Third, we consider optimizing the priorities of vessels for efficient use of resources berth and crane for the first time in BACAPs and employ <inline-formula><tex-math>$varepsilon$</tex-math></inline-formula>-constraints to search for feasible regions. Additionally, we develop a local search operator as a repair strategy to improve the quality of the solution. Finally, we test I2HCSO in a set of cases consisting of 25 BACAPs. Compared with the several typical optimizers with experimental results, I2HCSO is more competitive on BACAPs with different scales.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 2","pages":"1307-1321"},"PeriodicalIF":5.3,"publicationDate":"2025-02-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143716533","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Dual-Population Evolution Based Dynamic Constrained Multiobjective Optimization With Discontinuous and Irregular Feasible Regions 基于双种群进化的不连续不规则可行域动态约束多目标优化

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2025-02-18 DOI: 10.1109/TETCI.2025.3529882

Xiaoxu Jiang;Qingda Chen;Jinliang Ding;Xingyi Zhang

{"title":"Dual-Population Evolution Based Dynamic Constrained Multiobjective Optimization With Discontinuous and Irregular Feasible Regions","authors":"Xiaoxu Jiang;Qingda Chen;Jinliang Ding;Xingyi Zhang","doi":"10.1109/TETCI.2025.3529882","DOIUrl":"https://doi.org/10.1109/TETCI.2025.3529882","url":null,"abstract":"Dynamic constrained multiobjective optimization problems include irregular and discontinuous feasible regions, segmented true Pareto front, and dynamic environments. To address these problems, we design a dynamic constrained multiobjective optimization algorithm based on dual-population evolution. This algorithm includes two populations, P1 and P2, based on the feasibility of solutions. It utilizes valuable information from infeasible solutions to drive the populations toward the feasible regions and the true Pareto front. At the same time, we propose a mating selection operator to facilitate information exchange between populations and generate promising offspring solutions. To respond to environmental changes, we design a strategy that combines new solutions obtained by the sampling-selection-resampling method and updated old ones, rapidly generating a promising population in a new environment. Additionally, we also design a test suit that can effectively present the discontinuous feasible regions and the irregular changes of true Pareto front in practical appcation problems. The results from experiments demonstrate the efficacy of the test suit, and the proposed algorithm exhibits competitiveness compared to other algorithms.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 2","pages":"1352-1366"},"PeriodicalIF":5.3,"publicationDate":"2025-02-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143716520","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Population Stream-Driven Scalable Evolutionary Many-Objective Optimization 种群流驱动的可扩展进化多目标优化

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2025-02-17 DOI: 10.1109/TETCI.2025.3537916

Huangke Chen;Guohua Wu;Rui Wang;Witold Pedrycz

{"title":"Population Stream-Driven Scalable Evolutionary Many-Objective Optimization","authors":"Huangke Chen;Guohua Wu;Rui Wang;Witold Pedrycz","doi":"10.1109/TETCI.2025.3537916","DOIUrl":"https://doi.org/10.1109/TETCI.2025.3537916","url":null,"abstract":"Solving multi-objective optimization problems with scalable decision variables and objectives is an ongoing challenging task. This study proposes a new evolutionary framework that a series of continuously generated subpopulations are used to approximate the entire Pareto-optimal front. These dynamic subpopulations are abstracted as a population stream. In this framework, one subpopulation is only responsible for searching for a Pareto-optimal solution. Diversity is emphasized among converged solutions coming from different subpopulations, striving to alleviate the conflict between diversity and convergence. To improve the convergence of the newly generated subpopulations, the polynomial fitting method is performed on the obtained solutions to model the relationships among decision variables, which are then used to assist in the generation of new subpopulations. Moreover, an adaptive granularity grid-based environmental selection strategy is proposed to maintain a set of well-diversifying converged solutions. Lastly, extensive experiments are conducted to demonstrate the proposal's superiority by comparing it with 19 representative algorithms in 45 test instances with 3-15 objectives and 300-1500 decision variables.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 2","pages":"1406-1417"},"PeriodicalIF":5.3,"publicationDate":"2025-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143706666","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

AdaFML: Adaptive Federated Meta Learning With Multi-Objectives and Context-Awareness in Dynamic Heterogeneous Networks 动态异构网络中具有多目标和上下文感知的自适应联邦元学习

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2025-02-17 DOI: 10.1109/TETCI.2025.3537940

Qiaomei Han;Xianbin Wang;Weiming Shen;Yanjun Shi

{"title":"AdaFML: Adaptive Federated Meta Learning With Multi-Objectives and Context-Awareness in Dynamic Heterogeneous Networks","authors":"Qiaomei Han;Xianbin Wang;Weiming Shen;Yanjun Shi","doi":"10.1109/TETCI.2025.3537940","DOIUrl":"https://doi.org/10.1109/TETCI.2025.3537940","url":null,"abstract":"Recent advancements in Federated Learning (FL) have enabled the widespread deployment of distributed computing resources across connected devices, enhancing data processing capabilities and facilitating collaborative decision-making while maintaining user privacy. However, in Internet of Things (IoT) systems, the heterogeneity of devices and unstable network connections present significant challenges to the effective and efficient execution of FL tasks in real-world environments. To address these challenges, we propose an Adaptive Federated Meta Learning Framework with Multi-Objectives and Context-Awareness (AdaFML). This framework aims to achieve multiple objectives, including improving the performance of the FL global model, optimizing time efficiency, and enabling local model adaptation in dynamic and heterogeneous environments. Specifically, AdaFML extracts contextual information from each device, including its data distribution, computation, and communication conditions, to train a multimodal model that optimizes the FL task and time cost estimation, enhancing global model performance and time efficiency. Moreover, AdaFML fine-tunes two critical meta-learning parameters: the mixture ratio between local and global models and the selection weights for model aggregation. This enables adaptive local model updates across different devices while improving global model performance. Experimental results demonstrate that AdaFML boosts the effectiveness, efficiency, and adaptability of FL task execution in dynamic and heterogeneous environments.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 2","pages":"1428-1440"},"PeriodicalIF":5.3,"publicationDate":"2025-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143706633","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Towards the Explanation Consistency of Citizen Groups in Happiness Prediction via Factor Decorrelation 基于因子去相关的公民群体幸福感预测解释一致性研究

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2025-02-17 DOI: 10.1109/TETCI.2025.3537918

Xiaohua Wu;Lin Li;Xiaohui Tao;Jingling Yuan;Haoran Xie

{"title":"Towards the Explanation Consistency of Citizen Groups in Happiness Prediction via Factor Decorrelation","authors":"Xiaohua Wu;Lin Li;Xiaohui Tao;Jingling Yuan;Haoran Xie","doi":"10.1109/TETCI.2025.3537918","DOIUrl":"https://doi.org/10.1109/TETCI.2025.3537918","url":null,"abstract":"The happiness level of citizen groups has been widely analyzed using machine learning methods with explanation, aiming to support informed decision-making in our society. However, caused of complex correlations between happiness factors, there is inconsistency in case-by-case explanations provided by different models. In response, we propose a novel and trustworthy explanation solution for happiness prediction that can identify a broadly acceptable key factor set to improve explanation consistency across various models. First, the factor decorrelation is employed to ensure competitively high prediction accuracy. Second, we utilized a happiness prediction model pool that includes trained models with competitive accuracy, contributing to consistent explanations. The factor contribution is then computed using a post-hoc method based on the Shapley value with theoretical properties. The final key factor set is determined by the intersection of sets across different models. Experimental results using the Chinese General Social Survey (CGSS) and the European Social Survey (ESS) datasets validate the 2-fold increase in explanation consistency. Represented by specific citizen groups built on <monospace>age</monospace>, comprised of young group (<inline-formula><tex-math>$leq$</tex-math></inline-formula>40) and elder group (<inline-formula><tex-math>$>$</tex-math></inline-formula>40), and <monospace>health</monospace>, comprised of bad health (1-3) and good health (4-5), we demonstrate how these demographics exhibit different contributions in terms of factors. Additionally, we leverage four objective metrics to further evaluate the explanation quality and a human perspective metric for evaluating explanation consistency by comparing our results against explanatory and descriptive studies to provide qualitative reliability measures.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 2","pages":"1392-1405"},"PeriodicalIF":5.3,"publicationDate":"2025-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143706746","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Trustworthy Federated Fine-Tuning for Industrial Chains Demand Forecasting 面向产业链需求预测的可信联邦微调

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2025-02-13 DOI: 10.1109/TETCI.2025.3537941

Guoquan Huang;Guanyu Lin;Li Ning;Yicheng Xu;Chee Peng Lim;Yong Zhang

{"title":"Trustworthy Federated Fine-Tuning for Industrial Chains Demand Forecasting","authors":"Guoquan Huang;Guanyu Lin;Li Ning;Yicheng Xu;Chee Peng Lim;Yong Zhang","doi":"10.1109/TETCI.2025.3537941","DOIUrl":"https://doi.org/10.1109/TETCI.2025.3537941","url":null,"abstract":"Demand forecasting is crucial for the robust development of industrial chains, given the direct impact of consumer market volatility on production planning. However, in the intricate industrial chain environment, limited accessible data from independent production entities poses challenges in achieving high performances and precise predictions for future demand. Centralized training using machine learning modeling on data from multiple production entities is a potential solution, yet issues like consumer privacy, industry competition, and data security hinder practical machine learning implementation. This research introduces an innovative distributed learning approach, utilizing privacy-preserving federated learning techniques to enhance time-series demand forecasting for multiple entities pertaining to industrial chains. Our approach involves several key steps, including federated learning among entities in the industrial chain on a blockchain platform, ensuring the trustworthiness of the computation process and results. Leveraging Pre-training Models (PTMs) facilitates federated fine-tuning among production entities, addressing model heterogeneity and minimizing privacy breach risks. A comprehensive comparison study on various federated learning demand forecasting models on data from two real-world industry chains demonstrates the superior performance and enhanced security of our developed approach.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 2","pages":"1441-1453"},"PeriodicalIF":5.3,"publicationDate":"2025-02-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143706688","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Exploring the Adversarial Frontier: Quantifying Robustness via Adversarial Hypervolume 探索对抗性前沿：通过对抗性超容量量化稳健性

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2025-02-13 DOI: 10.1109/TETCI.2025.3535656

Ping Guo;Cheng Gong;Xi Lin;Zhiyuan Yang;Qingfu Zhang

{"title":"Exploring the Adversarial Frontier: Quantifying Robustness via Adversarial Hypervolume","authors":"Ping Guo;Cheng Gong;Xi Lin;Zhiyuan Yang;Qingfu Zhang","doi":"10.1109/TETCI.2025.3535656","DOIUrl":"https://doi.org/10.1109/TETCI.2025.3535656","url":null,"abstract":"The escalating threat of adversarial attacks on deep learning models, particularly in security-critical fields, has highlighted the need for robust deep learning systems. Conventional evaluation methods of their robustness rely on adversarial accuracy, which measures the model performance under a specific perturbation intensity. However, this singular metric does not fully encapsulate the overall resilience of a model against varying degrees of perturbation. To address this issue, we propose a new metric termed as the adversarial hypervolume for assessing the robustness of deep learning models comprehensively over a range of perturbation intensities from a multi-objective optimization standpoint. This metric allows for an in-depth comparison of defense mechanisms and recognizes the trivial improvements in robustness brought by less potent defensive strategies. We adopt a novel training algorithm to enhance adversarial robustness uniformly across various perturbation intensities, instead of only optimizing adversarial accuracy. Our experiments validate the effectiveness of the adversarial hypervolume metric in robustness evaluation, demonstrating its ability to reveal subtle differences in robustness that adversarial accuracy overlooks.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 2","pages":"1367-1378"},"PeriodicalIF":5.3,"publicationDate":"2025-02-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143716413","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Class Discriminative Knowledge Distillation 类判别知识蒸馏

IF 5.3 3区计算机科学

IEEE Transactions on Emerging Topics in Computational Intelligence Pub Date : 2025-01-29 DOI: 10.1109/TETCI.2025.3529896

Shuoxi Zhang;Hanpeng Liu;Yuyi Wang;Kun He;Jun Lin;Yang Zeng

{"title":"Class Discriminative Knowledge Distillation","authors":"Shuoxi Zhang;Hanpeng Liu;Yuyi Wang;Kun He;Jun Lin;Yang Zeng","doi":"10.1109/TETCI.2025.3529896","DOIUrl":"https://doi.org/10.1109/TETCI.2025.3529896","url":null,"abstract":"Knowledge distillation aims to transfer knowledge from a large teacher model to a lightweight student model, enabling the student to achieve performance comparable to the teacher. Existing methods explore various strategies for distillation, including soft logits, intermediate features, and even class-aware logits. Class-aware distillation, in particular, treats the columns of logit matrices as class representations, capturing potential relationships among instances within a batch. However, we argue that representing class embeddings solely as column vectors may not fully capture their inherent properties. In this study, we revisit class-aware knowledge distillation and propose that effective transfer of class-level knowledge requires two regularization strategies: <italic>separability and <italic>orthogonality. Additionally, we introduce an asymmetric architecture design to further enhance the transfer of class-level knowledge. Together, these components form a new methodology, Class Discriminative Knowledge Distillation (CD-KD). Empirical results demonstrate that CD-KD significantly outperforms several state-of-the-art logit-based and feature-based methods across diverse visual classification tasks, highlighting its effectiveness and robustness.","PeriodicalId":13135,"journal":{"name":"IEEE Transactions on Emerging Topics in Computational Intelligence","volume":"9 2","pages":"1340-1351"},"PeriodicalIF":5.3,"publicationDate":"2025-01-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143716404","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0