Byzantine-Resilient Decentralized Optimization for Joint Feature Selection in Multi-Task Networks

Abstract

This paper initiates a study of joint feature selection in multi-task networks, where agents collaboratively learn related prediction models over decentralized networks while keeping data local. Despite extensive progress in decentralized optimization and Byzantine-robust consensus, their integration to realize fully decentralized $\boldsymbol{\ell}_{\mathbf{2,1}}$-coupled multi-task feature learning remains an open challenge. To address this gap, the paper proposes a Byzantine-resilient decentralized proximal gradient framework, named Brd-MTFL, for joint feature selection and model learning under the $\boldsymbol{\ell}_{\mathbf{2,1}}$-regularization. The core innovation lies in a fully decentralized row-norm consensus mechanism that allows agents to compute global $\boldsymbol{\ell}_{\mathbf 2}$ row norms associated with feature groups through peer-to-peer communication, thus realizing fully decentralized joint feature selection. To counter malicious or faulty updates, a trimmed-mean-based robust aggregation rule is integrated into the row-norm estimation step, providing provable robustness to Byzantine failures. Our theoretical analysis establishes that Brd-MTFL achieves near-optimal convergence rates, thereby closing a critical theoretical gap at the intersection of Byzantine resilience, decentralized consensus on intermediate quantities, and $\boldsymbol{\ell}_{\mathbf{2,1}}$-coupled feature selection in multi-task networks. Specifically, we demonstrate finite-time $\boldsymbol{O}\mathbf{(1/}\boldsymbol{T}\mathbf{)}$ convergence under convexity and linear convergence under strong convexity, with both guarantees explicitly characterized by their dependence on network topology and the Byzantine budget. Empirical evaluations on synthetic data and real data demonstrate that Brd-MTFL achieves superior accuracy, stability, and resilience.