{"title":"Personalized Graph Federated Learning With Differential Privacy","authors":"Francois Gauthier;Vinay Chakravarthi Gogineni;Stefan Werner;Yih-Fang Huang;Anthony Kuh","doi":"10.1109/TSIPN.2023.3325963","DOIUrl":"https://doi.org/10.1109/TSIPN.2023.3325963","url":null,"abstract":"This paper presents a personalized graph federated learning (PGFL) framework in which distributedly connected servers and their respective edge devices collaboratively learn device or cluster-specific models while maintaining the privacy of every individual device. The proposed approach exploits similarities among different models to provide a more relevant experience for each device, even in situations with diverse data distributions and disproportionate datasets. Furthermore, to ensure a secure and efficient approach to collaborative personalized learning, we study a variant of the PGFL implementation that utilizes differential privacy, specifically zero-concentrated differential privacy, where a noise sequence perturbs model exchanges. Our mathematical analysis shows that the proposed privacy-preserving PGFL algorithm converges to the optimal cluster-specific solution for each cluster in linear time. It also reveals that exploiting similarities among clusters could lead to an alternative output whose distance to the original solution is bounded and that this bound can be adjusted by modifying the algorithm's hyperparameters. Further, our analysis shows that the algorithm ensures local differential privacy for all clients in terms of zero-concentrated differential privacy. Finally, the effectiveness of the proposed PGFL algorithm is showcased through numerical experiments conducted in the context of regression and classification tasks using some of the National Institute of Standards and Technology's (NIST's) datasets, namely, MNIST, and MedMNIST.","PeriodicalId":56268,"journal":{"name":"IEEE Transactions on Signal and Information Processing over Networks","volume":"9 ","pages":"736-749"},"PeriodicalIF":3.2,"publicationDate":"2023-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"109157459","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Leader-Following Containment Control of Hybrid Fractional-Order Networked Agents With Nonuniform Time Delays","authors":"Weihao Li;Lei Shi;Mengji Shi;Jiangfeng Yue;Boxian Lin;Kaiyu Qin","doi":"10.1109/TSIPN.2023.3325967","DOIUrl":"https://doi.org/10.1109/TSIPN.2023.3325967","url":null,"abstract":"Time delays, such as transmission delays or measurement delays, are common phenomena in practical networked control systems. These delays directly threaten the effective completion of cooperative tasks. In this study, the leader-following containment control problem of hybrid fractional-order networked agents with nonuniform time delays is addressed. The position and velocity loops of each double-integrator agent are modeled by fractional-order calculus equations of different orders, which is also called the hybrid fractional-order networked agent system. At first, the mathematical expressions for the upper bound of allowable time delays with respect to the system parameters, such as fractional order, topological structure properties, and controller gains, are given explicitly considering both the directed and undirected graph conditions. Then, this paper obtains the maximum allowable upper bounds of time delays for achieving leader-following containment tracking control in the case of fractional order mismatch. Based on this, it is convenient to calculate the delay margin directly and to judge the stability of the networked agent systems with nonuniform time delays. Finally, some simulation results are given to verify the effectiveness of the delay margin for networked agent systems. The results show that the system stability can be directly judged by calculating the critical time delay condition; meanwhile, the system robustness can also be improved by actively adjusting the controller parameters to increase the delay margin.","PeriodicalId":56268,"journal":{"name":"IEEE Transactions on Signal and Information Processing over Networks","volume":"9 ","pages":"750-760"},"PeriodicalIF":3.2,"publicationDate":"2023-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"109157520","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Noise Resilient Distributed Average Consensus Over Directed Graphs","authors":"Vivek Khatana;Murti V. Salapaka","doi":"10.1109/TSIPN.2023.3324583","DOIUrl":"https://doi.org/10.1109/TSIPN.2023.3324583","url":null,"abstract":"Motivated by the needs of resiliency, scalability, and plug-and-play operation, distributed decision making is becoming increasingly prevalent. The problem of achieving consensus in a multi-agent system is at the core of distributed decision making. In this article, we study the problem of achieving average consensus over a \u0000<italic>directed</i>\u0000 multi-agent network when the communication links are corrupted with \u0000<italic>noise</i>\u0000. We propose an algorithm where each agent updates its estimates based on the local mixture of information and adds its weighted noise-free initial information to its updates during every iteration. We demonstrate that, with appropriately designed weights, the agents achieve consensus despite additive communication noise. We establish that when the communication links are \u0000<italic>noiseless</i>\u0000, the proposed algorithm moves towards consensus at a geometric rate. Under communication noise, we prove that the agent estimates reach a consensus value \u0000<italic>almost surely</i>\u0000. We present numerical experiments to corroborate the efficacy of the proposed algorithm under different noise realizations and various algorithm parameters.","PeriodicalId":56268,"journal":{"name":"IEEE Transactions on Signal and Information Processing over Networks","volume":"9 ","pages":"770-785"},"PeriodicalIF":3.2,"publicationDate":"2023-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"109157521","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Time-Aware Distributed Sequential Detection of Gas Dispersion via Wireless Sensor Networks","authors":"Gianluca Tabella;Domenico Ciuonzo;Yasin Yilmaz;Xiaodong Wang;Pierluigi Salvo Rossi","doi":"10.1109/TSIPN.2023.3324586","DOIUrl":"https://doi.org/10.1109/TSIPN.2023.3324586","url":null,"abstract":"This work addresses the problem of detecting gas dispersions through concentration sensors with wireless transmission capabilities organized as a distributed Wireless Sensor Network (WSN). The concentration sensors in the WSN perform local sequential detection (SD) and transmit their individual decisions to the Fusion Center (FC) according to a transmission rule designed to meet the low-energy requirements of a wireless setup. The FC receives the transmissions sent by the sensors and makes a more reliable global decision by employing a SD algorithm. Two variants of the SD algorithm named \u0000<italic>Continuous Sampling Algorithm</i>\u0000 (CSA) and \u0000<italic>Decision-Triggered Sampling Algorithm</i>\u0000 (DTSA), each with its own transmission rule, are presented and compared against a fully-batch algorithm named \u0000<italic>Batch Sampling Algorithm</i>\u0000 (BSA). The CSA operates as a \u0000<italic>time-aware</i>\u0000 detector by incorporating the time of each transmission in the detection rule. The proposed framework encompasses the gas dispersion model into the FC's decision rule and leverages real-time weather measurements. The case study involves an accidental dispersion of carbon dioxide (CO\u0000<sub>2</sub>\u0000). System performances are evaluated in terms of the receiver operating characteristic (ROC) curve as well as average decision delay and communication cost.","PeriodicalId":56268,"journal":{"name":"IEEE Transactions on Signal and Information Processing over Networks","volume":"9 ","pages":"721-735"},"PeriodicalIF":3.2,"publicationDate":"2023-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"109229843","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Bakht Zaman;Luis Miguel Lopez-Ramos;Baltasar Beferull-Lozano
{"title":"Online Joint Topology Identification and Signal Estimation From Streams With Missing Data","authors":"Bakht Zaman;Luis Miguel Lopez-Ramos;Baltasar Beferull-Lozano","doi":"10.1109/TSIPN.2023.3324569","DOIUrl":"https://doi.org/10.1109/TSIPN.2023.3324569","url":null,"abstract":"Identifying the topology underlying a set of time series is useful for tasks such as prediction, denoising, and data completion. Vector autoregressive (VAR) model-based topologies capture dependencies among time series and are often inferred from observed spatio-temporal data. When data are affected by noise and/or missing samples, topology identification and signal recovery (reconstruction) tasks must be performed jointly. Additional challenges arise when i) the underlying topology is time-varying, ii) data become available sequentially, and iii) no delay is tolerated. This study proposes an online algorithm to overcome these challenges in estimating VAR model-based topologies, having constant complexity per iteration, which makes it interesting for big-data scenarios. The inexact proximal online gradient descent framework is used to derive a performance guarantee for the proposed algorithm, in the form of a dynamic regret bound. Numerical tests are also presented, showing the ability of the proposed algorithm to track time-varying topologies with missing data in an online fashion.","PeriodicalId":56268,"journal":{"name":"IEEE Transactions on Signal and Information Processing over Networks","volume":"9 ","pages":"691-704"},"PeriodicalIF":3.2,"publicationDate":"2023-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"109157460","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Vinay Chakravarthi Gogineni;Ashkan Moradi;Naveen K. D. Venkategowda;Stefan Werner
{"title":"Communication-Efficient and Privacy-Aware Distributed Learning","authors":"Vinay Chakravarthi Gogineni;Ashkan Moradi;Naveen K. D. Venkategowda;Stefan Werner","doi":"10.1109/TSIPN.2023.3322783","DOIUrl":"https://doi.org/10.1109/TSIPN.2023.3322783","url":null,"abstract":"Communication efficiency and privacy are two key concerns in modern distributed computing systems. Towards this goal, this article proposes partial sharing private distributed learning (PPDL) algorithms that offer communication efficiency while preserving privacy, thus making them suitable for applications with limited resources in adversarial environments. First, we propose a noise injection-based PPDL algorithm that achieves communication efficiency by sharing only a fraction of the information at each consensus iteration and provides privacy by perturbing the information exchanged among neighbors. To further increase privacy, local information is randomly decomposed into private and public substates before sharing with the neighbors. This results in a decomposition- and noise-injection-based PPDL strategy in which only a freaction of the perturbeesd public substate is shared during local collaborations, whereas the private substate is updated locally without being shared. To determine the impact of communication savings and privacy preservation on the performance of distributed learning algorithms, we analyze the mean and mean-square convergence of the proposed algorithms. Moreover, we investigate the privacy of agents by characterizing privacy as the mean squared error of the estimate of private information at the honest-but-curious adversary. The analytical results show a tradeoff between communication efficiency and privacy in proposed PPDL algorithms, while decomposition- and noise-injection-based PPDL improves privacy compared to noise-injection-based PPDL. Lastly, numerical simulations corroborate the analytical findings.","PeriodicalId":56268,"journal":{"name":"IEEE Transactions on Signal and Information Processing over Networks","volume":"9 ","pages":"705-720"},"PeriodicalIF":3.2,"publicationDate":"2023-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"109157458","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Distributed Saddle Point Problems for Strongly Concave-Convex Functions","authors":"Muhammad I. Qureshi;Usman A. Khan","doi":"10.1109/TSIPN.2023.3317807","DOIUrl":"https://doi.org/10.1109/TSIPN.2023.3317807","url":null,"abstract":"In this article, we propose \u0000<monospace><b>GT-GDA</b></monospace>\u0000, a distributed optimization method to solve saddle point problems of the form: \u0000<inline-formula><tex-math>${min _{mathbf {x}} max _{mathbf {y}} lbrace F(mathbf x,mathbf y) :=G(mathbf x) + langle mathbf y, overline{P} mathbf x rangle - H(mathbf y) rbrace }$</tex-math></inline-formula>\u0000, where the functions \u0000<inline-formula><tex-math>$G(cdot)$</tex-math></inline-formula>\u0000, \u0000<inline-formula><tex-math>$H(cdot)$</tex-math></inline-formula>\u0000, and the coupling matrix \u0000<inline-formula><tex-math>$overline{P}$</tex-math></inline-formula>\u0000 are distributed over a strongly connected network of nodes. \u0000<monospace><b>GT-GDA</b></monospace>\u0000 is a first-order method that uses gradient tracking to eliminate the dissimilarity caused by heterogeneous data distribution among the nodes. In the most general form, \u0000<monospace><b>GT-GDA</b></monospace>\u0000 includes a consensus over the local coupling matrices to achieve the optimal (unique) saddle point, however, at the expense of increased communication. To avoid this, we propose a more efficient variant \u0000<monospace><b>GT-GDA-Lite</b></monospace>\u0000 that does not incur additional communication and analyze its convergence in various scenarios. We show that \u0000<monospace><b>GT-GDA</b></monospace>\u0000 converges linearly to the unique saddle point solution when \u0000<inline-formula><tex-math>$G$</tex-math></inline-formula>\u0000 is smooth and convex, \u0000<inline-formula><tex-math>$H$</tex-math></inline-formula>\u0000 is smooth and strongly convex, and the global coupling matrix \u0000<inline-formula><tex-math>$overline{P}$</tex-math></inline-formula>\u0000 has full column rank. We further characterize the regime under which \u0000<monospace><b>GT-GDA</b></monospace>\u0000 exhibits a network topology-independent convergence behavior. We next show the linear convergence of \u0000<monospace><b>GT-GDA-Lite</b></monospace>\u0000 to an error around the unique saddle point, which goes to zero when the coupling cost \u0000<inline-formula><tex-math>${langle mathbf y, overline{P} mathbf x rangle }$</tex-math></inline-formula>\u0000 is common to all nodes, or when \u0000<inline-formula><tex-math>$G$</tex-math></inline-formula>\u0000 and \u0000<inline-formula><tex-math>$H$</tex-math></inline-formula>\u0000 are quadratic. Numerical experiments illustrate the convergence properties and importance of \u0000<monospace><b>GT-GDA</b></monospace>\u0000 and \u0000<monospace><b>GT-GDA-Lite</b></monospace>\u0000 for several applications.","PeriodicalId":56268,"journal":{"name":"IEEE Transactions on Signal and Information Processing over Networks","volume":"9 ","pages":"679-690"},"PeriodicalIF":3.2,"publicationDate":"2023-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"67863374","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Joint Multi-Ground-User Edge Caching Resource Allocation for Cache-Enabled High-Low-Altitude-Platforms Integrated Network","authors":"Yongyi Yuan;Enchang Sun;Hanxing Qu","doi":"10.1109/TSIPN.2023.3315597","DOIUrl":"https://doi.org/10.1109/TSIPN.2023.3315597","url":null,"abstract":"This article examines the cache-enabled high-low-altitude-platforms integrated network (CHLIN), which consists of multiple high-altitude platforms (HAPs) and cacheable low-altitude platforms (LAPs). CHLIN aims to leverage the edge caching, the flexibility of LAPs and the broad coverage and stability of HAPs to realize multi-ground-user content transmission. Considering the low endurance, dynamics, and limited storage capacity of LAPs, a combined optimization of content caching policies, offloading decisions, and HAP-servers and LAP-servers selection is designed to reduce the delay of content transmission while fulfilling users' demand for the quality of service. We transform the complex non-convex optimization problem with highly coupled variables into an equivalent convex problem. Afterward, a genetic-algorithm-embedded distributed alternating direction method of multipliers (GA-DADMM) is proposed, which adopts a distributed architecture for alternating iteration and introduces a genetic algorithm to derive the multi-dimensional and coupled local variables. Simulation results show that GA-DADMM achieves better convergence than the comparison algorithm, which is proper for large-scale optimization problems. The superiority of the proposed edge caching scheme in transmission delay reduction is also validated.","PeriodicalId":56268,"journal":{"name":"IEEE Transactions on Signal and Information Processing over Networks","volume":"9 ","pages":"655-668"},"PeriodicalIF":3.2,"publicationDate":"2023-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"67863245","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Frequency-Domain Diffusion Bias-Compensated Adaptation With Periodic Communication","authors":"Yishu Peng;Sheng Zhang;Hongyang Chen;Zhengchun Zhou;Xiaohu Tang","doi":"10.1109/TSIPN.2023.3313810","DOIUrl":"https://doi.org/10.1109/TSIPN.2023.3313810","url":null,"abstract":"When the input signal of each node is interfered by noise, the distributed frequency-domain adaptive algorithm yields biased estimation. To eliminate the noise-induced bias with reduced communication load, this article proposes the frequency-domain diffusion bias-compensated adaptive filtering with periodic communication. By minimizing the bias-eliminating cost function, the frequency-domain diffusion bias-compensated LMS (FD-BCLMS) is first derived. Subsequently, to achieve lower computational complexity and communication cost, we design the double periodic FD-BCLMS (DPFD-BCLMS) algorithm by resorting to periodic update and communication strategies. Moreover, the DPFD-BCLMS with power normalized scheme (DPFD-BCNLMS) is developed to improve the convergence rate in the case of colored input. The transient and steady-state behaviors are investigated. For the steady-state performance degradation in the DPFD-BCNLMS, we modify the combination step near steady-state, resulting in the switched DPFD-BCNLMS (SDPFD-BCNLMS). A new estimation method for the input noise variance is also provided. Finally, the superiority of the proposed algorithms is validated by numerical simulations.","PeriodicalId":56268,"journal":{"name":"IEEE Transactions on Signal and Information Processing over Networks","volume":"9 ","pages":"626-639"},"PeriodicalIF":3.2,"publicationDate":"2023-09-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"67864364","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Temporal Multiple Rotation Averaging on a Distributed Dynamic Network","authors":"Aidan Blair;Amirali Khodadadian Gostar;Ruwan Tennakoon;Alireza Bab-Hadiashar;Reza Hoseinnezhad","doi":"10.1109/TSIPN.2023.3313817","DOIUrl":"https://doi.org/10.1109/TSIPN.2023.3313817","url":null,"abstract":"This article proposes a solution for multiple rotation averaging on time-series data such as video. In applications using video data such as target tracking, in addition to the data found in individual frames, temporal information across multiple frames such as target trajectories can be used to more accurately estimate target states. Existing techniques for robust rotation averaging, including traditional iterative optimization and emerging neural network methods, do not exploit this temporal information. We first introduce the problem of using temporal data in rotation averaging and propose an extension to existing multiple rotation averaging methods via temporal rrotations. We then propose implementing a motion model for the cameras and predicting camera states using a particle filter, which are used to initialize the rotation averaging algorithm. These methods' performance is evaluated through a Monte Carlo Simulation on synthetic data and compared to an existing method. The results show that using temporal data in time-series datasets significantly increases the accuracy compared to the traditional algorithm for rotation averaging.","PeriodicalId":56268,"journal":{"name":"IEEE Transactions on Signal and Information Processing over Networks","volume":"9 ","pages":"669-678"},"PeriodicalIF":3.2,"publicationDate":"2023-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"67863246","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}