{"title":"Robust In-Memory Computation With Bayesian Analog Error Mitigating Codes","authors":"Nilesh Kumar Jha;Huayan Guo;Vincent K. N. Lau","doi":"10.1109/TSP.2025.3530149","DOIUrl":"10.1109/TSP.2025.3530149","url":null,"abstract":"In-memory computation (IMC) is a promising technology for enabling low-latency and energy-efficient deep learning and artificial intelligence (AI) applications at edge devices. However, the IMC crossbar array, typically implemented using resistive random access memory (RRAM), faces hardware defects that pose a significant challenge to reliable computation. This paper presents a robust IMC scheme utilizing Bayesian neural network-accelerated analog codes. Our approach includes a new datapath design comprising a parity matrix generator and a low-complexity decoder module to facilitate analog codes for IMC. Moreover, we introduce a Gaussian mixture model-based error prior to capture impulsive error statistics and leverage variational Bayesian inference (VBI) techniques for training neural network weights. Extensive simulations confirm the effectiveness of our proposed solution compared to various state-of-the-art baseline schemes.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"534-548"},"PeriodicalIF":4.6,"publicationDate":"2025-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142987669","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ming Xiang;Stratis Ioannidis;Edmund Yeh;Carlee Joe-Wong;Lili Su
{"title":"Empowering Federated Learning With Implicit Gossiping: Mitigating Connection Unreliability Amidst Unknown and Arbitrary Dynamics","authors":"Ming Xiang;Stratis Ioannidis;Edmund Yeh;Carlee Joe-Wong;Lili Su","doi":"10.1109/TSP.2025.3526782","DOIUrl":"10.1109/TSP.2025.3526782","url":null,"abstract":"Federated learning is a popular distributed learning approach for training a machine learning model without disclosing raw data. It consists of a parameter server and a possibly large collection of clients (e.g., in cross-device federated learning) that may operate in congested and changing environments. In this paper, we study federated learning in the presence of stochastic and dynamic communication failures wherein the uplink between the parameter server and client <inline-formula><tex-math>$i$</tex-math></inline-formula> is on with <italic>unknown</i> probability <inline-formula><tex-math>$p_{i}^{t}$</tex-math></inline-formula> in round <inline-formula><tex-math>$t$</tex-math></inline-formula>. Furthermore, we allow the dynamics of <inline-formula><tex-math>$p_{i}^{t}$</tex-math></inline-formula> to be <italic>arbitrary</i>. We first demonstrate that when the <inline-formula><tex-math>$p_{i}^{t}$</tex-math></inline-formula>'s vary across clients, the most widely adopted federated learning algorithm, Federated Average (FedAvg), experiences significant bias. To address this observation, we propose Federated Postponed Broadcast (FedPBC), a simple variant of FedAvg. It differs from FedAvg in that the parameter server postpones broadcasting the global model to the clients with active uplinks till the end of each training round. Despite uplink failures, we show that FedPBC converges to a stationary point of the original non-convex objective. On the technical front, postponing the global model broadcasts enables implicit gossiping among the clients with active links in round <inline-formula><tex-math>$t$</tex-math></inline-formula>. In spite of the time-varying nature of <inline-formula><tex-math>$p_{i}^{t}$</tex-math></inline-formula>, we can bound the perturbation of the global model dynamics using techniques to control gossip-type information mixing errors. Extensive experiments have been conducted on real-world datasets over diversified unreliable uplink patterns to corroborate our analysis.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"766-780"},"PeriodicalIF":4.6,"publicationDate":"2025-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142987243","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Optimal Beamforming for MIMO DFRC Systems With Transmit Covariance Constraints","authors":"Chenhao Yang;Xin Wang;Wei Ni;Yi Jiang","doi":"10.1109/TSP.2025.3529722","DOIUrl":"10.1109/TSP.2025.3529722","url":null,"abstract":"This paper optimizes the beamforming design of a downlink multiple-input multiple-output (MIMO) dual-function radar-communication (DFRC) system to maximize the weighted communication sum-rate under a prescribed transmit covariance constraint for radar performance guarantee. In the single-user case, we show that the transmit covariance constraint implies the existence of inherent orthogonality among the transmit beamforming vectors in use. Then, leveraging Cauchy's interlace theorem, we derive the globally optimal transmit and receive beamforming solution in closed form. In the multi-user case, we exploit the connection between the weighted sum-rate and weighted minimum mean squared error (MMSE) to reformulate the intended problem, and develop a block-coordinate-descent (BCD) algorithm to iteratively compute the transmit beamforming and receive beamforming solutions. Under this approach, we reveal that the optimal receive beamforming is given by the classic MMSE one and the optimal transmit beamforming design amounts to solving an orthogonal Procrustes problem, thereby allowing for closed-form solutions to subproblems in each BCD step and fast convergence of the proposed algorithm to a high-quality (near-optimal) overall beamforming design. Numerical results demonstrate the superiority of our approach to the existing methods, with at least 40% higher sum-rate under a multi-user MIMO setting in the high signal-to-noise regime.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"601-616"},"PeriodicalIF":4.6,"publicationDate":"2025-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142986646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Bo Tang;Da Li;Wenjun Wu;Astha Saini;Prabhu Babu;Petre Stoica
{"title":"Dual-Function Beamforming Design for Multi-Target Localization and Reliable Communications","authors":"Bo Tang;Da Li;Wenjun Wu;Astha Saini;Prabhu Babu;Petre Stoica","doi":"10.1109/TSP.2025.3529950","DOIUrl":"10.1109/TSP.2025.3529950","url":null,"abstract":"This paper investigates the transmit beamforming design for multiple-input multiple-output systems to support both multi-target localization and multi-user communications. To enhance the target localization performance, we derive the asymptotic Cramér-Rao bound (CRB) for target angle estimation by assuming that the receive array is linear and uniform. Then we formulate a beamforming design problem based on minimizing an upper bound on the asymptotic CRB (which is shown to be equivalent to maximizing the harmonic mean of the weighted beampattern responses at the target directions). Moreover, we impose a constraint on the SINR of each received communication signal to guarantee reliable communication performance. Two iterative algorithms are derived to tackle the non-convex design problem: one is based on the alternating direction method of multipliers, and the other uses the majorization-minimization technique to solve an equivalent minimax problem. Numerical results show that, through elaborate dual-function beamforming matrix design, the proposed algorithms can simultaneously achieve superior angle estimation performance as well as high-quality multi-user communications.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"559-573"},"PeriodicalIF":4.6,"publicationDate":"2025-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142981500","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Shuning Sun;Dongpo Xu;Qiankun Diao;Danilo P. Mandic
{"title":"Cramér-Rao Lower Bounds for Unconstrained and Constrained Quaternion Parameters","authors":"Shuning Sun;Dongpo Xu;Qiankun Diao;Danilo P. Mandic","doi":"10.1109/TSP.2025.3529468","DOIUrl":"10.1109/TSP.2025.3529468","url":null,"abstract":"The Cramér-Rao lower bound (CRLB) is a fundamental result in statistical signal processing, however, the CRLB for quaternion parameters is not yet established. To this end, we develop the theory of quaternion Cramér-Rao lower bound (QCRLB), based on the generalized Hamilton-real (GHR) calculus. For generality, this is achieved in a way that conforms with the real and complex CRLB. We first provide the properties of the quaternion covariance matrix and the quaternion Fisher information matrix (FIM), paving the way for the derivation of the QCRLB. This serves as a basis for the formulation of the QCRLB without constraints and a criterion for determining whether the QCRLB is attained. We also establish the QCRLB for constrained quaternion parameters, including both nonsingular and singular cases of the quaternion FIM. These broaden the theoretical framework and enhance its applicability to diverse practical scenarios. The practical efficacy of the QCRLB is demonstrated through two illustrative examples. Numerical validations confirm that the maximum-likelihood estimator (MLE) attains the QCRLB for the linear model, and the quaternion gradient ascent (QGA) algorithm achieves the QCRLB at each iteration with theoretical guarantees. We also propose the quaternion constrained scoring (QCS) algorithm, which converges in one step in the linear constrained MLE case, for the linear model. These results significantly contribute to both the theory and practical application of quaternion signal processing, bringing valuable insights into the quaternion parameter estimation.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"508-518"},"PeriodicalIF":4.6,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142974993","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Unraveling the Viral Spread of Misinformation: Maximum-Likelihood Estimation and Starlike Tree Approximation in Markovian Spreading Models","authors":"Pei-Duo Yu;Chee Wei Tan","doi":"10.1109/TSP.2025.3527755","DOIUrl":"10.1109/TSP.2025.3527755","url":null,"abstract":"Identifying the source of epidemic-like spread in networks is crucial for removing internet viruses or finding the source of rumors in online social networks. The challenge lies in tracing the source from a snapshot observation of infected nodes. How do we accurately pinpoint the source? Utilizing snapshot data, we apply a probabilistic approach, focusing on the graph boundary and the observed time, to detect sources via an effective maximum likelihood algorithm. A novel starlike tree approximation extends applicability to general graphs, demonstrating versatility. Unlike previous works that rely heavily on structural properties alone, our method also incorporates temporal data for more precise source detection. We highlight the utility of the Gamma function for analyzing the ratio of the likelihood being the source between nodes asymptotically. Comprehensive evaluations confirm algorithmic effectiveness in diverse network scenarios, advancing source detection in large-scale network analysis and information dissemination strategies.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"446-461"},"PeriodicalIF":4.6,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142974841","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lin Chen;Xue Jiang;Pei Xiao;Xingzhao Liu;Martin Haardt
{"title":"Structured Tensor Decomposition for FDD Massive MIMO Downlink Channel Reconstruction","authors":"Lin Chen;Xue Jiang;Pei Xiao;Xingzhao Liu;Martin Haardt","doi":"10.1109/TSP.2025.3529657","DOIUrl":"10.1109/TSP.2025.3529657","url":null,"abstract":"The downlink channel reconstruction at the base station holds paramount significance across a multitude of applications in FDD massive MIMO systems. Conventional approaches rely on downlink training and feedback with a considerable overhead. In order to mitigate this issue, we propose a tensor-based framework for downlink channel reconstruction that leverages the partial reciprocity between the uplink and downlink channels. By modeling the uplink channel as a multi-dimensional tensor, we estimate the reciprocal channel parameters via a low-rank tensor decomposition. This approach effectively captures the correlation between arrays, subcarriers, and polarizations of the channel. In addition to the classical tensor decomposition, we exploit the exponential structure of the decomposed antenna and delay steering matrices, and propose a structured tensor decomposition algorithm. The proposed algorithm enhances the exponential structure via a tensor rank-1 constraint by incorporating the Hankel transform. The resulting optimization problem is rendered tractable by introducing a domain conversion matrix to facilitate the mapping of variables between the Hankel transform domain and the original domain. The proposed method exhibits superior noise robustness compared to conventional algebraic closed-form methods based on the Vandermonde constrained tensor decomposition. Experimental results with both simulated data and a Ray-tracing dataset demonstrate the effectiveness and superior downlink reconstruction accuracy of our proposed methods compared with several alternative approaches.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"887-902"},"PeriodicalIF":4.6,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142974994","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Solving Quadratic Systems With Full-Rank Matrices Using Sparse or Generative Priors","authors":"Junren Chen;Michael K. Ng;Zhaoqiang Liu","doi":"10.1109/TSP.2024.3522179","DOIUrl":"10.1109/TSP.2024.3522179","url":null,"abstract":"The problem of recovering a signal <inline-formula><tex-math>$boldsymbol{x}inmathbb{R}^{n}$</tex-math></inline-formula> from a quadratic system <inline-formula><tex-math>${y_{i}=boldsymbol{x}^{top}boldsymbol{A}_{i}boldsymbol{x}, i=1,ldots,m}$</tex-math></inline-formula> with full-rank matrices <inline-formula><tex-math>$boldsymbol{A}_{i}$</tex-math></inline-formula> frequently arises in applications such as unassigned distance geometry and sub-wavelength imaging. With i.i.d. standard Gaussian matrices <inline-formula><tex-math>$boldsymbol{A}_{i}$</tex-math></inline-formula>, this paper addresses the high-dimensional case where <inline-formula><tex-math>$mll n$</tex-math></inline-formula> by incorporating prior knowledge of <inline-formula><tex-math>$boldsymbol{x}$</tex-math></inline-formula>. First, we consider a <inline-formula><tex-math>$k$</tex-math></inline-formula>-sparse <inline-formula><tex-math>$boldsymbol{x}$</tex-math></inline-formula> and introduce the thresholded Wirtinger flow (TWF) algorithm that does not require the sparsity level <inline-formula><tex-math>$k$</tex-math></inline-formula>. TWF comprises two steps: the spectral initialization that identifies a point sufficiently close to <inline-formula><tex-math>$boldsymbol{x}$</tex-math></inline-formula> (up to a sign flip) when <inline-formula><tex-math>$m=O(k^{2}log n)$</tex-math></inline-formula>, and the thresholded gradient descent which, when provided a good initialization, produces a sequence linearly converging to <inline-formula><tex-math>$boldsymbol{x}$</tex-math></inline-formula> with <inline-formula><tex-math>$m=O(klog n)$</tex-math></inline-formula> measurements. Second, we explore the generative prior, assuming that <inline-formula><tex-math>$boldsymbol{x}$</tex-math></inline-formula> lies in the range of an <inline-formula><tex-math>$L$</tex-math></inline-formula>-Lipschitz continuous generative model with <inline-formula><tex-math>$k$</tex-math></inline-formula>-dimensional inputs in an <inline-formula><tex-math>$ell_{2}$</tex-math></inline-formula>-ball of radius <inline-formula><tex-math>$r$</tex-math></inline-formula>. With an estimate correlated with the signal, we develop the projected gradient descent (PGD) algorithm that also comprises two steps: the projected power method that provides an initial vector with <inline-formula><tex-math>$Obig{(}sqrt{klog(L)/m}big{)}$</tex-math></inline-formula> <inline-formula><tex-math>$ell_{2}$</tex-math></inline-formula>-error given <inline-formula><tex-math>$m=O(klog(Lnr))$</tex-math></inline-formula> measurements, and the projected gradient descent that refines the <inline-formula><tex-math>$ell_{2}$</tex-math></inline-formula>-error to <inline-formula><tex-math>$O(delta)$</tex-math></inline-formula> at a geometric rate when <inline-formula><tex-math>$m=O(klogfrac{Lrn}{delta^{2}})$</tex-math></inline-formula>. Experimental results corroborate our theoretical findings and show that: (i) our approach for the sparse case nota","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"477-492"},"PeriodicalIF":4.6,"publicationDate":"2025-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142961228","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Multi-Fidelity Bayesian Optimization With Across-Task Transferable Max-Value Entropy Search","authors":"Yunchuan Zhang;Sangwoo Park;Osvaldo Simeone","doi":"10.1109/TSP.2025.3528252","DOIUrl":"10.1109/TSP.2025.3528252","url":null,"abstract":"In many applications, ranging from logistics to engineering, a designer is faced with a sequence of optimization tasks for which the objectives are in the form of black-box functions that are costly to evaluate. Furthermore, higher-fidelity evaluations of the optimization objectives often entail a larger cost. Existing multi-fidelity black-box optimization strategies select candidate solutions and fidelity levels with the goal of maximizing the information about the optimal value or the optimal solution for the current task. Assuming that successive optimization tasks are related, this paper introduces a novel information-theoretic acquisition function that balances the need to acquire information about the current task with the goal of collecting information transferable to future tasks. The proposed method transfers across tasks distributions over parameters of a Gaussian process surrogate model by implementing particle-based variational Bayesian updates. Theoretical insights based on the analysis of the expected regret substantiate the benefits of acquiring transferable knowledge across tasks. Furthermore, experimental results across synthetic and real-world examples reveal that the proposed acquisition strategy that caters to future tasks can significantly improve the optimization efficiency as soon as a sufficient number of tasks is processed.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"418-432"},"PeriodicalIF":4.6,"publicationDate":"2025-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142961197","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Adaptive Polyak Step-Size for Momentum Accelerated Stochastic Gradient Descent With General Convergence Guarantee","authors":"Jiawei Zhang;Cheng Jin;Yuantao Gu","doi":"10.1109/TSP.2025.3528217","DOIUrl":"10.1109/TSP.2025.3528217","url":null,"abstract":"Momentum accelerated stochastic gradient descent (SGDM) has gained significant popularity in several signal processing and machine learning tasks. Despite its widespread success, the step size of SGDM remains a critical hyperparameter affecting its performance and often requires manual tuning. Recently, some works have introduced the Polyak step size to SGDM and provided corresponding convergence analysis. However, the convergence guarantee of existing Polyak step sizes for SGDM are limited to convex objectives and lack theoretical support for more widely applicable non-convex problems. To bridge this gap, we design a novel Polyak adaptive step size for SGDM. The proposed algorithm, termed SGDM-APS, incorporates a moving average form tailored for the momentum mechanism in SGDM. We establish the convergence guarantees of SGDM-APS for both convex and non-convex objectives, providing theoretical analysis of its effectiveness. To the best of our knowledge, SGDM-APS is the first Polyak step size for SGDM with general convergence guarantee. Our analysis can also be extended to constant step size SGDM, enriching the theoretical comprehension of the classic SGDM algorithm. Through extensive experiments on diverse benchmarks, we demonstrate that SGDM-APS achieves competitive convergence rates and generalization performance compared to several popular optimization algorithms.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"462-476"},"PeriodicalIF":4.6,"publicationDate":"2025-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142961606","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}