{"title":"A Deterministic–Probabilistic Approach to Neural Network Pruning","authors":"Soumyadipta Banerjee;Jiaul H. Paik","doi":"10.1109/TAI.2025.3558718","DOIUrl":"https://doi.org/10.1109/TAI.2025.3558718","url":null,"abstract":"Modern deep networks are highly over-parameterized. Thus, training and testing such models in various applications are computationally intensive with excessive memory and energy requirements. Network pruning aims to find smaller subnetworks from within these dense networks that do not compromise on the test accuracy. In this article, we present a probabilistic and deterministic pruning methodology which determines the likelihood of retention of the weight parameters by modeling the layer-specific distribution of extreme values of the weights. Our method automatically finds the sparsity in each layer, unlike existing pruning techniques which require an explicit input of the sparsity information. Experiments in the present work show that deterministic–probabilistic pruning consistently achieves high sparsity levels, ranging from 65 to 95%, while maintaining comparable or improved testing accuracy across multiple datasets such as MNIST, CIFAR-10, and Tiny ImageNet, on architectures including VGG-16, ResNet-18, and ResNet-50.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 10","pages":"2830-2839"},"PeriodicalIF":0.0,"publicationDate":"2025-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145196043","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Malicious Clients and Contribution Co-Aware Federated Unlearning","authors":"Yang Wang;Xue Li;Siguang Chen","doi":"10.1109/TAI.2025.3556092","DOIUrl":"https://doi.org/10.1109/TAI.2025.3556092","url":null,"abstract":"Existing federated unlearning methods to eliminate the negative impact of malicious clients on the global model are influenced by unreasonable assumptions (e.g., an auxiliary dataset) or fail to balance model performance and efficiency. To overcome these shortcomings, we propose a malicious clients and contribution co-aware federated unlearning (MCC-Fed) method. Specifically, we introduce a method for detecting malicious clients to reduce their impact on the global model. Next, we design a contribution-aware metric, which accurately quantifies the negative impact of malicious clients on the global calculating their historical contribution ratio. Then, based on this metric, we propose a novel federated unlearning method in which benign clients use the contribution-aware metric as a regularization term to unlearn the influence of malicious clients, and restoring model performance. Experimental results demonstrate that our method effectively addresses the issue of excessive unlearning during the unlearning process, improves the efficiency of performance recovery, and enhances robustness against malicious clients. Federated unlearning effectively removes malicious clients’ influence while reducing training costs compared to retraining.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 10","pages":"2848-2857"},"PeriodicalIF":0.0,"publicationDate":"2025-03-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145196041","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Maximum Margin-Based Activation Clipping for Posttraining Overfitting Mitigation in DNN Classifiers","authors":"Hang Wang;David J. Miller;George Kesidis","doi":"10.1109/TAI.2025.3552686","DOIUrl":"https://doi.org/10.1109/TAI.2025.3552686","url":null,"abstract":"Sources of overfitting in deep neural net (DNN) classifiers include: 1) large class imbalances; 2) insufficient training set diversity; and 3) over-training. Recently, it was shown that backdoor data-poisoning <italic>also</i> induces overfitting, with unusually large maximum classification margins (MMs) to the attacker’s target class. This is enabled by (unbounded) ReLU activation functions, which allow large signals to propagate in the DNN. Thus, an effective <italic>posttraining</i> backdoor mitigation approach (with no knowledge of the training set and no knowledge or control of the training process) was proposed, informed by a small, clean (poisoning-free) data set and choosing saturation levels on neural activations to limit the DNN’s MMs. Here, we show that nonmalicious sources of overfitting <italic>also</i> exhibit unusually large MMs. Thus, we propose novel posttraining MM-based regularization that substantially mitigates <italic>nonmalicious</i> overfitting due to class imbalances and overtraining. Whereas backdoor mitigation and other adversarial learning defenses often <italic>trade off</i> a classifier’s accuracy to achieve robustness against attacks, our approach, inspired by ideas from adversarial learning, <italic>helps</i> the classifier’s generalization accuracy: as shown for CIFAR-10 and CIFAR-100, our approach improves both the accuracy for rare categories as well as overall. Moreover, unlike other overfitting mitigation methods, it does so with no knowledge of class imbalances, no knowledge of the training set, and without control of the training process.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 10","pages":"2840-2847"},"PeriodicalIF":0.0,"publicationDate":"2025-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145196042","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Learning From N-Tuple Similarities and Unlabeled Data","authors":"Junpeng Li;Shuying Huang;Changchun Hua;Yana Yang","doi":"10.1109/TAI.2025.3552687","DOIUrl":"https://doi.org/10.1109/TAI.2025.3552687","url":null,"abstract":"Learning from pairwise similarity and unlabeled data (SU) is a recently emerging weakly-supervised learning method, which learns a classifier from similar data pairs (two instances belonging to the same class) and unlabeled data. However, this framework is insoluble for triplet similarities and unlabeled data. To address this limitation, this article develops a framework for learning from triplet similarities (three instances belonging to the same class) and unlabeled data points, denoted as TSU. This framework not only showcases the feasibility of constructing a TSU classifier but also serves as an inspiration to explore the broader challenge of addressing N-tuple similarities (N ≥ 2) and unlabeled data points. To tackle this more generalized problem, the present article develops an advancing weakly-supervision framework of learning from N-tuple similarities (N instances belong to the same class) and unlabeled data points, named NSU. This framework provides a solid foundation for handling diverse similarity scenarios. Based on these findings, we propose empirical risk minimization estimators for both TSU and NSU classification. The estimation error bounds are also established for the proposed methods. Finally, experiments are performed to verify the effectiveness of the proposed algorithm.","PeriodicalId":73305,"journal":{"name":"IEEE transactions on artificial intelligence","volume":"6 9","pages":"2542-2551"},"PeriodicalIF":0.0,"publicationDate":"2025-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144926901","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
求助内容：
应助结果提醒方式：
京公网安备 11010802042870号
