{"title":"FL-Joint: joint aligning features and labels in federated learning for data heterogeneity","authors":"Wenxin Chen, Jinrui Zhang, Deyu Zhang","doi":"10.1007/s40747-024-01636-4","DOIUrl":null,"url":null,"abstract":"<p>Federated learning is a distributed machine learning paradigm that trains a shared model using data from various clients, it faces a core challenge in data heterogeneity arising from diverse client settings and environments. Existing methods typically focus on weight divergence mitigation and aggregation strategy enhancements, they overlook the mixed skew in label and feature distributions prevalent in real-world data. To address this, we present <b><i>FL-Joint</i></b>, a federated learning framework that aligns label and feature distributions using auxiliary loss functions. This framework involves a class-balanced classifier as the local model. It aligns label and feature distributions locally by using auxiliary loss functions based on class-conditional information and pseudo-labels. This alignment drives client feature distributions to converge towards a shared feature space, refining decision boundaries and boosting the global model’s generalization ability. Extensive experiments across diverse datasets and heterogeneous data settings show that our method significantly improves accuracy and convergence speed compared to baseline approaches.</p>","PeriodicalId":10524,"journal":{"name":"Complex & Intelligent Systems","volume":"6 1","pages":""},"PeriodicalIF":5.0000,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Complex & Intelligent Systems","FirstCategoryId":"94","ListUrlMain":"https://doi.org/10.1007/s40747-024-01636-4","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
Abstract
Federated learning is a distributed machine learning paradigm that trains a shared model using data from various clients, it faces a core challenge in data heterogeneity arising from diverse client settings and environments. Existing methods typically focus on weight divergence mitigation and aggregation strategy enhancements, they overlook the mixed skew in label and feature distributions prevalent in real-world data. To address this, we present FL-Joint, a federated learning framework that aligns label and feature distributions using auxiliary loss functions. This framework involves a class-balanced classifier as the local model. It aligns label and feature distributions locally by using auxiliary loss functions based on class-conditional information and pseudo-labels. This alignment drives client feature distributions to converge towards a shared feature space, refining decision boundaries and boosting the global model’s generalization ability. Extensive experiments across diverse datasets and heterogeneous data settings show that our method significantly improves accuracy and convergence speed compared to baseline approaches.
期刊介绍:
Complex & Intelligent Systems aims to provide a forum for presenting and discussing novel approaches, tools and techniques meant for attaining a cross-fertilization between the broad fields of complex systems, computational simulation, and intelligent analytics and visualization. The transdisciplinary research that the journal focuses on will expand the boundaries of our understanding by investigating the principles and processes that underlie many of the most profound problems facing society today.