Xujing Li;Sheng Sun;Min Liu;Ju Ren;Xuefeng Jiang;Tianliu He
{"title":"FedCRAC: Improving Federated Classification Performance on Long-Tailed Data via Classifier Representation Adjustment and Calibration","authors":"Xujing Li;Sheng Sun;Min Liu;Ju Ren;Xuefeng Jiang;Tianliu He","doi":"10.1109/TMC.2024.3466208","DOIUrl":null,"url":null,"abstract":"Federated learning has been a popular distributed training paradigm that enables to train a shared model with data privacy protection. However, non-Independent Identically Distribution and long-tailed data distribution characteristics across mobile devices results in evident performance degradation, especially for classification tasks. Although plenty of research studies devote to alleviating classification performance degradation caused by highly-skewed data distribution, they still cannot improve the distinguishability of model representation on hard-to-learn tail classes, and face obvious divergence of local classifiers in FL setting. To this end, we propose Federated Classifier Representation Adjustment and Calibration to improve the representation distinguishability of tail classes and achieve inter-client representation alignment with acceptable resource consumption on attaching operations. We first design a Class Similarity-Aware Margin matrix to enlarge class representation discrepancy and improve local classifier discriminability on tail classes during client-side local training process. To mitigate the divergence of local classifiers across clients, we further propose the Self Distillation Classifier Calibration to achieve the aggregated global classifier calibration with the assistance of generated pseudo representation samples via self-distillation manner. We conduct various experiments under wide-range long-tailed and heterogeneous data settings. Experimental results show that FedCRAC outperforms state-of-the-art methods in terms of accuracy and resource consumption.","PeriodicalId":50389,"journal":{"name":"IEEE Transactions on Mobile Computing","volume":"24 1","pages":"482-499"},"PeriodicalIF":7.7000,"publicationDate":"2024-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Mobile Computing","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10689340/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
Federated learning has been a popular distributed training paradigm that enables to train a shared model with data privacy protection. However, non-Independent Identically Distribution and long-tailed data distribution characteristics across mobile devices results in evident performance degradation, especially for classification tasks. Although plenty of research studies devote to alleviating classification performance degradation caused by highly-skewed data distribution, they still cannot improve the distinguishability of model representation on hard-to-learn tail classes, and face obvious divergence of local classifiers in FL setting. To this end, we propose Federated Classifier Representation Adjustment and Calibration to improve the representation distinguishability of tail classes and achieve inter-client representation alignment with acceptable resource consumption on attaching operations. We first design a Class Similarity-Aware Margin matrix to enlarge class representation discrepancy and improve local classifier discriminability on tail classes during client-side local training process. To mitigate the divergence of local classifiers across clients, we further propose the Self Distillation Classifier Calibration to achieve the aggregated global classifier calibration with the assistance of generated pseudo representation samples via self-distillation manner. We conduct various experiments under wide-range long-tailed and heterogeneous data settings. Experimental results show that FedCRAC outperforms state-of-the-art methods in terms of accuracy and resource consumption.
期刊介绍:
IEEE Transactions on Mobile Computing addresses key technical issues related to various aspects of mobile computing. This includes (a) architectures, (b) support services, (c) algorithm/protocol design and analysis, (d) mobile environments, (e) mobile communication systems, (f) applications, and (g) emerging technologies. Topics of interest span a wide range, covering aspects like mobile networks and hosts, mobility management, multimedia, operating system support, power management, online and mobile environments, security, scalability, reliability, and emerging technologies such as wearable computers, body area networks, and wireless sensor networks. The journal serves as a comprehensive platform for advancements in mobile computing research.