IEEE Transactions on Signal Processing最新文献_第8页

Adaptive Model Pruning and Personalization for Federated Learning Over Wireless Networks 无线网络联合学习的自适应模型剪枝和个性化设计

IF 4.6 2区工程技术

IEEE Transactions on Signal Processing Pub Date : 2024-09-11 DOI: 10.1109/TSP.2024.3459808

Xiaonan Liu;Tharmalingam Ratnarajah;Mathini Sellathurai;Yonina C. Eldar

{"title":"Adaptive Model Pruning and Personalization for Federated Learning Over Wireless Networks","authors":"Xiaonan Liu;Tharmalingam Ratnarajah;Mathini Sellathurai;Yonina C. Eldar","doi":"10.1109/TSP.2024.3459808","DOIUrl":"10.1109/TSP.2024.3459808","url":null,"abstract":"Federated learning (FL) enables distributed learning across edge devices while protecting data privacy. However, the learning accuracy decreases due to the heterogeneity of devices’ data, and the computation and communication latency increase when updating large-scale learning models on devices with limited computational capability and wireless resources. We consider a FL framework with partial model pruning and personalization to overcome these challenges. This framework splits the learning model into a global part with model pruning shared with all devices to learn data representations and a personalized part to be fine-tuned for a specific device, which adapts the model size during FL. Our approach reduces both computation and communication latency and increases the learning accuracy for devices with non-independent and identically distributed data. The computation and communication latency and convergence of the proposed FL framework are mathematically analyzed. To maximize the convergence rate and guarantee learning accuracy, Karush–Kuhn–Tucker (KKT) conditions are deployed to jointly optimize the pruning ratio and bandwidth allocation. Finally, experimental results demonstrate that the proposed FL framework achieves similar learning accuracy and a remarkable reduction of approximately \u0000<inline-formula><tex-math>$50%$</tex-math></inline-formula>\u0000 computation and communication latency compared with FL with partial model personalization.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4395-4411"},"PeriodicalIF":4.6,"publicationDate":"2024-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142170619","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Extreme Point Pursuit—Part II: Further Error Bound Analysis and Applications 极值点追寻--第二部分：进一步的误差边界分析和应用

IF 4.6 2区工程技术

IEEE Transactions on Signal Processing Pub Date : 2024-09-11 DOI: 10.1109/TSP.2024.3458015

Junbin Liu;Ya Liu;Wing-Kin Ma;Mingjie Shao;Anthony Man-Cho So

引用次数: 0

Robust Wideband Beampattern Synthesis With Precise Control of Worst-Case Beampattern 通过精确控制最坏情况贝型实现稳健的宽带贝型合成

IF 4.6 2区工程技术

IEEE Transactions on Signal Processing Pub Date : 2024-09-10 DOI: 10.1109/TSP.2024.3457159

Congwei Feng;Huawei Chen

{"title":"Robust Wideband Beampattern Synthesis With Precise Control of Worst-Case Beampattern","authors":"Congwei Feng;Huawei Chen","doi":"10.1109/TSP.2024.3457159","DOIUrl":"10.1109/TSP.2024.3457159","url":null,"abstract":"Beampattern synthesis inspired by adaptive array theory (AAT) has attracted much interest in recent years, thanks to its capability to flexibly and precisely control beampattern. However, the existing AAT-inspired beampattern synthesis approaches usually assume an ideal array model, which is not realistic in practice and may lead to severe performance degradation in the presence of steering vector errors. In this paper, we propose a robust beampattern synthesis approach for wideband arrays using regularized AAT-inspired weighted least squares (WLS), which can precisely control the worst-case beampattern, including both its mainlobe ripple and sidelobe level, in the presence of steering vector errors. We develop a theory on the solutions for the regularization parameter and weighting function of the regularized AAT-inspired WLS. We propose a Newton-Raphson method to find the solution for the regularization parameter, and derive closed-form solutions for the weighting function. Moreover, we also offer some insight into the effect of steering vector errors on the control of worst-case beampattern. The effectiveness of the proposed algorithm is verified by design examples, including robust synthesis of frequency-invariant and flat-top wideband beampatterns.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4573-4588"},"PeriodicalIF":4.6,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142166021","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Principal Component Analysis in Space Forms 空间形式的主成分分析

IF 4.6 2区工程技术

IEEE Transactions on Signal Processing Pub Date : 2024-09-10 DOI: 10.1109/TSP.2024.3457529

Puoya Tabaghi;Michael Khanzadeh;Yusu Wang;Siavash Mirarab

{"title":"Principal Component Analysis in Space Forms","authors":"Puoya Tabaghi;Michael Khanzadeh;Yusu Wang;Siavash Mirarab","doi":"10.1109/TSP.2024.3457529","DOIUrl":"10.1109/TSP.2024.3457529","url":null,"abstract":"Principal Component Analysis (PCA) is a workhorse of modern data science. While PCA assumes the data conforms to Euclidean geometry, for specific data types, such as hierarchical and cyclic data structures, other spaces are more appropriate. We study PCA in space forms; that is, those with constant curvatures. At a point on a Riemannian manifold, we can define a Riemannian affine subspace based on a set of tangent vectors. Finding the optimal low-dimensional affine subspace for given points in a space form amounts to dimensionality reduction. Our Space Form PCA (SFPCA) seeks the affine subspace that best represents a set of manifold-valued points with the minimum projection cost. We propose proper cost functions that enjoy two properties: (1) their optimal affine subspace is the solution to an eigenequation, and (2) optimal affine subspaces of different dimensions form a nested set. These properties provide advances over existing methods, which are mostly iterative algorithms with slow convergence and weaker theoretical guarantees. We evaluate the proposed SFPCA on real and simulated data in spherical and hyperbolic spaces. We show that it outperforms alternative methods in estimating true subspaces (in simulated data) with respect to convergence speed or accuracy, often both.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4428-4443"},"PeriodicalIF":4.6,"publicationDate":"2024-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10670424","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142166477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Modeling and State Estimation of Destination-Constrained Dynamic Systems. Part II: Uncertain Arrival Time 目的地受限动态系统的建模与状态估计。第二部分：不确定到达时间

IF 5.4 2区工程技术

IEEE Transactions on Signal Processing Pub Date : 2024-09-05 DOI: 10.1109/tsp.2024.3454972

Linfeng Xu, X. Rong Li, Mahendra Mallick, Zhansheng Duan

引用次数: 0

Ordinary Differential Equation-Based MIMO Signal Detection 基于常微分方程的多输入多输出信号检测

IF 4.6 2区工程技术

IEEE Transactions on Signal Processing Pub Date : 2024-09-05 DOI: 10.1109/TSP.2024.3454703

Ayano Nakai-Kasai;Tadashi Wadayama

引用次数: 0

Revisiting High-Order Tensor Singular Value Decomposition From Basic Element Perspective 从基本元素视角重温高阶张量奇异值分解

IF 4.6 2区工程技术

IEEE Transactions on Signal Processing Pub Date : 2024-09-04 DOI: 10.1109/TSP.2024.3454115

Sheng Liu;Xi-Le Zhao;Jinsong Leng;Ben-Zheng Li;Jing-Hua Yang;Xinyu Chen

{"title":"Revisiting High-Order Tensor Singular Value Decomposition From Basic Element Perspective","authors":"Sheng Liu;Xi-Le Zhao;Jinsong Leng;Ben-Zheng Li;Jing-Hua Yang;Xinyu Chen","doi":"10.1109/TSP.2024.3454115","DOIUrl":"10.1109/TSP.2024.3454115","url":null,"abstract":"Recently, tensor singular value decomposition (t-SVD), based on the tensor-tensor product (t-product), has become a powerful tool for processing third-order tensor data. However, constrained by the fact that the basic element is the fiber (i.e., vector) in the t-product, higher-order tensor data (i.e., order \u0000<inline-formula><tex-math>$d>3$</tex-math></inline-formula>\u0000) are usually unfolded into third-order tensors to satisfy the classical t-product setting, which leads to the destroying of high-dimensional structure. By revisiting the basic element in the t-product, we suggest a generalized t-product called element-based tensor-tensor product (elt-product) as an alternative of the classic t-product, where the basic element is a \u0000<inline-formula><tex-math>$(d-2)$</tex-math></inline-formula>\u0000th-order tensor instead of a vector. The benefit of the elt-product is that it can better preserve high-dimensional structures and that it can explore more complex interactions via higher-order convolution instead of first-order convolution in classic t-product. Starting from the elt-product, we develop new tensor-SVD and low-rank tensor metrics (e.g., rank and nuclear norm). Equipped with the suggested metrics, we present a tensor completion model for high-order tensor data and prove the exact recovery guarantees. To harness the resulting nonconvex optimization problem, we apply an alternating direction method of the multiplier (ADMM) algorithm with a theoretical convergence guarantee. Extensive experimental results on the simulated and real-world data (color videos, light-field images, light-field videos, and traffic data) demonstrate the superiority of the proposed model against the state-of-the-art baseline models.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4589-4603"},"PeriodicalIF":4.6,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137969","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Convex Parameter Estimation of Perturbed Multivariate Generalized Gaussian Distributions 受扰动多变量广义高斯分布的凸参数估计

IF 4.6 2区工程技术

IEEE Transactions on Signal Processing Pub Date : 2024-09-04 DOI: 10.1109/TSP.2024.3453509

Nora Ouzir;Frédéric Pascal;Jean-Christophe Pesquet

{"title":"Convex Parameter Estimation of Perturbed Multivariate Generalized Gaussian Distributions","authors":"Nora Ouzir;Frédéric Pascal;Jean-Christophe Pesquet","doi":"10.1109/TSP.2024.3453509","DOIUrl":"10.1109/TSP.2024.3453509","url":null,"abstract":"The multivariate generalized Gaussian distribution (MGGD), also known as the multivariate exponential power (MEP) distribution, is widely used in signal and image processing. However, estimating MGGD parameters, which is required in practical applications, still faces specific theoretical challenges. In particular, establishing convergence properties for the standard fixed-point approach when both the distribution mean and the scatter (or the precision) matrix are unknown is still an open problem. In robust estimation, imposing classical constraints on the precision matrix, such as sparsity, has been limited by the non-convexity of the resulting cost function. This paper tackles these issues from an optimization viewpoint by proposing a convex formulation with well-established convergence properties. We embed our analysis in a noisy scenario where robustness is induced by modelling multiplicative perturbations. The resulting framework is flexible as it combines a variety of regularizations for the precision matrix, the mean and model perturbations. This paper presents proof of the desired theoretical properties, specifies the conditions preserving these properties for different regularization choices and designs a general proximal primal-dual optimization strategy. The experiments show a more accurate precision and covariance matrix estimation with similar performance for the mean vector parameter compared to Tyler's \u0000<inline-formula><tex-math>$M$</tex-math></inline-formula>\u0000-estimator. In a high-dimensional setting, the proposed method outperforms the classical GLASSO, one of its robust extensions, and the regularized Tyler's estimator.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4132-4146"},"PeriodicalIF":4.6,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137942","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Cramér-Rao Bound for Signal Parameter Estimation From Modulo ADC Generated Data 根据模数转换器生成的数据进行信号参数估计的 Cramér-Rao 边界

IF 4.6 2区工程技术

IEEE Transactions on Signal Processing Pub Date : 2024-09-03 DOI: 10.1109/TSP.2024.3453346

Yuanbo Cheng;Johan Karlsson;Jian Li

引用次数: 0

Real-Time Transfer Active Learning for Functional Regression and Prediction Based on Multi-Output Gaussian Process 基于多输出高斯过程的功能回归与预测的实时转移主动学习

IF 4.6 2区工程技术

IEEE Transactions on Signal Processing Pub Date : 2024-09-02 DOI: 10.1109/TSP.2024.3451412

Zengchenghao Xia;Zhiyong Hu;Qingbo He;Chao Wang

{"title":"Real-Time Transfer Active Learning for Functional Regression and Prediction Based on Multi-Output Gaussian Process","authors":"Zengchenghao Xia;Zhiyong Hu;Qingbo He;Chao Wang","doi":"10.1109/TSP.2024.3451412","DOIUrl":"10.1109/TSP.2024.3451412","url":null,"abstract":"Active learning provides guidance for the design and modeling of systems with highly expensive sampling costs. However, existing active learning approaches suffer from cold-start concerns, where the performance is impaired due to the initial few experiments designed by active learning. In this paper, we propose using transfer learning to solve the cold-start problem of functional regression by leveraging knowledge from related and data-rich signals to achieve robust and superior performance, especially when only a few experiments are available in the signal of interest. More specifically, we construct a multi-output Gaussian process (MGP) to model the between-signal functional relationship. This MGP features unique innovations that distinguish the proposed transfer active learning from existing works: i) a specially designed covariance structure is proposed for characterizing within-and between-signal inter-relationships and facilitating interpretable transfer learning, and ii) an iterative Bayesian framework is proposed to update the parameters and prediction of the MGP in real-time, which significantly reduces the computational load and facilitates the iterative active learning. The inter-relationship captured by this novel MGP is then fed into active learning using the integrated mean-squared error (IMSE) as the objective. We provide theoretical justifications for this active learning mechanism, which demonstrates the objective (IMSE) is monotonically decreasing as we gather more data from the proposed transfer active learning. The real-time updating and the monotonically decreasing objective together provide both practical efficiency and theoretical guarantees for solving the cold-start concern in active learning. The proposed method is compared with benchmark methods through various numerical and real case studies, and the results demonstrate the superiority of the method, especially when limited experiments are available at the initial stage of design.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"72 ","pages":"4163-4177"},"PeriodicalIF":4.6,"publicationDate":"2024-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142123822","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0