IEEE Transactions on Signal Processing最新文献

筛选
英文 中文
Robust In-Memory Computation With Bayesian Analog Error Mitigating Codes 基于贝叶斯模拟错误缓解码的鲁棒内存计算
IF 4.6 2区 工程技术
IEEE Transactions on Signal Processing Pub Date : 2025-01-16 DOI: 10.1109/TSP.2025.3530149
Nilesh Kumar Jha;Huayan Guo;Vincent K. N. Lau
{"title":"Robust In-Memory Computation With Bayesian Analog Error Mitigating Codes","authors":"Nilesh Kumar Jha;Huayan Guo;Vincent K. N. Lau","doi":"10.1109/TSP.2025.3530149","DOIUrl":"10.1109/TSP.2025.3530149","url":null,"abstract":"In-memory computation (IMC) is a promising technology for enabling low-latency and energy-efficient deep learning and artificial intelligence (AI) applications at edge devices. However, the IMC crossbar array, typically implemented using resistive random access memory (RRAM), faces hardware defects that pose a significant challenge to reliable computation. This paper presents a robust IMC scheme utilizing Bayesian neural network-accelerated analog codes. Our approach includes a new datapath design comprising a parity matrix generator and a low-complexity decoder module to facilitate analog codes for IMC. Moreover, we introduce a Gaussian mixture model-based error prior to capture impulsive error statistics and leverage variational Bayesian inference (VBI) techniques for training neural network weights. Extensive simulations confirm the effectiveness of our proposed solution compared to various state-of-the-art baseline schemes.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"534-548"},"PeriodicalIF":4.6,"publicationDate":"2025-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142987669","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Empowering Federated Learning With Implicit Gossiping: Mitigating Connection Unreliability Amidst Unknown and Arbitrary Dynamics 通过隐性八卦增强联邦学习:在未知和任意动态中减轻连接不可靠性
IF 4.6 2区 工程技术
IEEE Transactions on Signal Processing Pub Date : 2025-01-16 DOI: 10.1109/TSP.2025.3526782
Ming Xiang;Stratis Ioannidis;Edmund Yeh;Carlee Joe-Wong;Lili Su
{"title":"Empowering Federated Learning With Implicit Gossiping: Mitigating Connection Unreliability Amidst Unknown and Arbitrary Dynamics","authors":"Ming Xiang;Stratis Ioannidis;Edmund Yeh;Carlee Joe-Wong;Lili Su","doi":"10.1109/TSP.2025.3526782","DOIUrl":"10.1109/TSP.2025.3526782","url":null,"abstract":"Federated learning is a popular distributed learning approach for training a machine learning model without disclosing raw data. It consists of a parameter server and a possibly large collection of clients (e.g., in cross-device federated learning) that may operate in congested and changing environments. In this paper, we study federated learning in the presence of stochastic and dynamic communication failures wherein the uplink between the parameter server and client <inline-formula><tex-math>$i$</tex-math></inline-formula> is on with <italic>unknown</i> probability <inline-formula><tex-math>$p_{i}^{t}$</tex-math></inline-formula> in round <inline-formula><tex-math>$t$</tex-math></inline-formula>. Furthermore, we allow the dynamics of <inline-formula><tex-math>$p_{i}^{t}$</tex-math></inline-formula> to be <italic>arbitrary</i>. We first demonstrate that when the <inline-formula><tex-math>$p_{i}^{t}$</tex-math></inline-formula>'s vary across clients, the most widely adopted federated learning algorithm, Federated Average (FedAvg), experiences significant bias. To address this observation, we propose Federated Postponed Broadcast (FedPBC), a simple variant of FedAvg. It differs from FedAvg in that the parameter server postpones broadcasting the global model to the clients with active uplinks till the end of each training round. Despite uplink failures, we show that FedPBC converges to a stationary point of the original non-convex objective. On the technical front, postponing the global model broadcasts enables implicit gossiping among the clients with active links in round <inline-formula><tex-math>$t$</tex-math></inline-formula>. In spite of the time-varying nature of <inline-formula><tex-math>$p_{i}^{t}$</tex-math></inline-formula>, we can bound the perturbation of the global model dynamics using techniques to control gossip-type information mixing errors. Extensive experiments have been conducted on real-world datasets over diversified unreliable uplink patterns to corroborate our analysis.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"766-780"},"PeriodicalIF":4.6,"publicationDate":"2025-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142987243","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Optimal Beamforming for MIMO DFRC Systems With Transmit Covariance Constraints 具有发射协方差约束的MIMO DFRC系统的最优波束形成
IF 4.6 2区 工程技术
IEEE Transactions on Signal Processing Pub Date : 2025-01-15 DOI: 10.1109/TSP.2025.3529722
Chenhao Yang;Xin Wang;Wei Ni;Yi Jiang
{"title":"Optimal Beamforming for MIMO DFRC Systems With Transmit Covariance Constraints","authors":"Chenhao Yang;Xin Wang;Wei Ni;Yi Jiang","doi":"10.1109/TSP.2025.3529722","DOIUrl":"10.1109/TSP.2025.3529722","url":null,"abstract":"This paper optimizes the beamforming design of a downlink multiple-input multiple-output (MIMO) dual-function radar-communication (DFRC) system to maximize the weighted communication sum-rate under a prescribed transmit covariance constraint for radar performance guarantee. In the single-user case, we show that the transmit covariance constraint implies the existence of inherent orthogonality among the transmit beamforming vectors in use. Then, leveraging Cauchy's interlace theorem, we derive the globally optimal transmit and receive beamforming solution in closed form. In the multi-user case, we exploit the connection between the weighted sum-rate and weighted minimum mean squared error (MMSE) to reformulate the intended problem, and develop a block-coordinate-descent (BCD) algorithm to iteratively compute the transmit beamforming and receive beamforming solutions. Under this approach, we reveal that the optimal receive beamforming is given by the classic MMSE one and the optimal transmit beamforming design amounts to solving an orthogonal Procrustes problem, thereby allowing for closed-form solutions to subproblems in each BCD step and fast convergence of the proposed algorithm to a high-quality (near-optimal) overall beamforming design. Numerical results demonstrate the superiority of our approach to the existing methods, with at least 40% higher sum-rate under a multi-user MIMO setting in the high signal-to-noise regime.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"601-616"},"PeriodicalIF":4.6,"publicationDate":"2025-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142986646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Dual-Function Beamforming Design for Multi-Target Localization and Reliable Communications 多目标定位与可靠通信的双功能波束形成设计
IF 4.6 2区 工程技术
IEEE Transactions on Signal Processing Pub Date : 2025-01-14 DOI: 10.1109/TSP.2025.3529950
Bo Tang;Da Li;Wenjun Wu;Astha Saini;Prabhu Babu;Petre Stoica
{"title":"Dual-Function Beamforming Design for Multi-Target Localization and Reliable Communications","authors":"Bo Tang;Da Li;Wenjun Wu;Astha Saini;Prabhu Babu;Petre Stoica","doi":"10.1109/TSP.2025.3529950","DOIUrl":"10.1109/TSP.2025.3529950","url":null,"abstract":"This paper investigates the transmit beamforming design for multiple-input multiple-output systems to support both multi-target localization and multi-user communications. To enhance the target localization performance, we derive the asymptotic Cramér-Rao bound (CRB) for target angle estimation by assuming that the receive array is linear and uniform. Then we formulate a beamforming design problem based on minimizing an upper bound on the asymptotic CRB (which is shown to be equivalent to maximizing the harmonic mean of the weighted beampattern responses at the target directions). Moreover, we impose a constraint on the SINR of each received communication signal to guarantee reliable communication performance. Two iterative algorithms are derived to tackle the non-convex design problem: one is based on the alternating direction method of multipliers, and the other uses the majorization-minimization technique to solve an equivalent minimax problem. Numerical results show that, through elaborate dual-function beamforming matrix design, the proposed algorithms can simultaneously achieve superior angle estimation performance as well as high-quality multi-user communications.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"559-573"},"PeriodicalIF":4.6,"publicationDate":"2025-01-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142981500","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Cramér-Rao Lower Bounds for Unconstrained and Constrained Quaternion Parameters 无约束和约束四元数参数的cram<s:1> - rao下界
IF 4.6 2区 工程技术
IEEE Transactions on Signal Processing Pub Date : 2025-01-13 DOI: 10.1109/TSP.2025.3529468
Shuning Sun;Dongpo Xu;Qiankun Diao;Danilo P. Mandic
{"title":"Cramér-Rao Lower Bounds for Unconstrained and Constrained Quaternion Parameters","authors":"Shuning Sun;Dongpo Xu;Qiankun Diao;Danilo P. Mandic","doi":"10.1109/TSP.2025.3529468","DOIUrl":"10.1109/TSP.2025.3529468","url":null,"abstract":"The Cramér-Rao lower bound (CRLB) is a fundamental result in statistical signal processing, however, the CRLB for quaternion parameters is not yet established. To this end, we develop the theory of quaternion Cramér-Rao lower bound (QCRLB), based on the generalized Hamilton-real (GHR) calculus. For generality, this is achieved in a way that conforms with the real and complex CRLB. We first provide the properties of the quaternion covariance matrix and the quaternion Fisher information matrix (FIM), paving the way for the derivation of the QCRLB. This serves as a basis for the formulation of the QCRLB without constraints and a criterion for determining whether the QCRLB is attained. We also establish the QCRLB for constrained quaternion parameters, including both nonsingular and singular cases of the quaternion FIM. These broaden the theoretical framework and enhance its applicability to diverse practical scenarios. The practical efficacy of the QCRLB is demonstrated through two illustrative examples. Numerical validations confirm that the maximum-likelihood estimator (MLE) attains the QCRLB for the linear model, and the quaternion gradient ascent (QGA) algorithm achieves the QCRLB at each iteration with theoretical guarantees. We also propose the quaternion constrained scoring (QCS) algorithm, which converges in one step in the linear constrained MLE case, for the linear model. These results significantly contribute to both the theory and practical application of quaternion signal processing, bringing valuable insights into the quaternion parameter estimation.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"508-518"},"PeriodicalIF":4.6,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142974993","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Unraveling the Viral Spread of Misinformation: Maximum-Likelihood Estimation and Starlike Tree Approximation in Markovian Spreading Models 解开错误信息的病毒式传播:马尔可夫传播模型中的最大似然估计和星形树近似
IF 4.6 2区 工程技术
IEEE Transactions on Signal Processing Pub Date : 2025-01-13 DOI: 10.1109/TSP.2025.3527755
Pei-Duo Yu;Chee Wei Tan
{"title":"Unraveling the Viral Spread of Misinformation: Maximum-Likelihood Estimation and Starlike Tree Approximation in Markovian Spreading Models","authors":"Pei-Duo Yu;Chee Wei Tan","doi":"10.1109/TSP.2025.3527755","DOIUrl":"10.1109/TSP.2025.3527755","url":null,"abstract":"Identifying the source of epidemic-like spread in networks is crucial for removing internet viruses or finding the source of rumors in online social networks. The challenge lies in tracing the source from a snapshot observation of infected nodes. How do we accurately pinpoint the source? Utilizing snapshot data, we apply a probabilistic approach, focusing on the graph boundary and the observed time, to detect sources via an effective maximum likelihood algorithm. A novel starlike tree approximation extends applicability to general graphs, demonstrating versatility. Unlike previous works that rely heavily on structural properties alone, our method also incorporates temporal data for more precise source detection. We highlight the utility of the Gamma function for analyzing the ratio of the likelihood being the source between nodes asymptotically. Comprehensive evaluations confirm algorithmic effectiveness in diverse network scenarios, advancing source detection in large-scale network analysis and information dissemination strategies.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"446-461"},"PeriodicalIF":4.6,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142974841","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Structured Tensor Decomposition for FDD Massive MIMO Downlink Channel Reconstruction 结构化张量分解用于FDD海量MIMO下行信道重构
IF 4.6 2区 工程技术
IEEE Transactions on Signal Processing Pub Date : 2025-01-13 DOI: 10.1109/TSP.2025.3529657
Lin Chen;Xue Jiang;Pei Xiao;Xingzhao Liu;Martin Haardt
{"title":"Structured Tensor Decomposition for FDD Massive MIMO Downlink Channel Reconstruction","authors":"Lin Chen;Xue Jiang;Pei Xiao;Xingzhao Liu;Martin Haardt","doi":"10.1109/TSP.2025.3529657","DOIUrl":"10.1109/TSP.2025.3529657","url":null,"abstract":"The downlink channel reconstruction at the base station holds paramount significance across a multitude of applications in FDD massive MIMO systems. Conventional approaches rely on downlink training and feedback with a considerable overhead. In order to mitigate this issue, we propose a tensor-based framework for downlink channel reconstruction that leverages the partial reciprocity between the uplink and downlink channels. By modeling the uplink channel as a multi-dimensional tensor, we estimate the reciprocal channel parameters via a low-rank tensor decomposition. This approach effectively captures the correlation between arrays, subcarriers, and polarizations of the channel. In addition to the classical tensor decomposition, we exploit the exponential structure of the decomposed antenna and delay steering matrices, and propose a structured tensor decomposition algorithm. The proposed algorithm enhances the exponential structure via a tensor rank-1 constraint by incorporating the Hankel transform. The resulting optimization problem is rendered tractable by introducing a domain conversion matrix to facilitate the mapping of variables between the Hankel transform domain and the original domain. The proposed method exhibits superior noise robustness compared to conventional algebraic closed-form methods based on the Vandermonde constrained tensor decomposition. Experimental results with both simulated data and a Ray-tracing dataset demonstrate the effectiveness and superior downlink reconstruction accuracy of our proposed methods compared with several alternative approaches.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"887-902"},"PeriodicalIF":4.6,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142974994","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Solving Quadratic Systems With Full-Rank Matrices Using Sparse or Generative Priors 利用稀疏先验或生成先验求解具有全秩矩阵的二次系统
IF 4.6 2区 工程技术
IEEE Transactions on Signal Processing Pub Date : 2025-01-10 DOI: 10.1109/TSP.2024.3522179
Junren Chen;Michael K. Ng;Zhaoqiang Liu
{"title":"Solving Quadratic Systems With Full-Rank Matrices Using Sparse or Generative Priors","authors":"Junren Chen;Michael K. Ng;Zhaoqiang Liu","doi":"10.1109/TSP.2024.3522179","DOIUrl":"10.1109/TSP.2024.3522179","url":null,"abstract":"The problem of recovering a signal &lt;inline-formula&gt;&lt;tex-math&gt;$boldsymbol{x}inmathbb{R}^{n}$&lt;/tex-math&gt;&lt;/inline-formula&gt; from a quadratic system &lt;inline-formula&gt;&lt;tex-math&gt;${y_{i}=boldsymbol{x}^{top}boldsymbol{A}_{i}boldsymbol{x}, i=1,ldots,m}$&lt;/tex-math&gt;&lt;/inline-formula&gt; with full-rank matrices &lt;inline-formula&gt;&lt;tex-math&gt;$boldsymbol{A}_{i}$&lt;/tex-math&gt;&lt;/inline-formula&gt; frequently arises in applications such as unassigned distance geometry and sub-wavelength imaging. With i.i.d. standard Gaussian matrices &lt;inline-formula&gt;&lt;tex-math&gt;$boldsymbol{A}_{i}$&lt;/tex-math&gt;&lt;/inline-formula&gt;, this paper addresses the high-dimensional case where &lt;inline-formula&gt;&lt;tex-math&gt;$mll n$&lt;/tex-math&gt;&lt;/inline-formula&gt; by incorporating prior knowledge of &lt;inline-formula&gt;&lt;tex-math&gt;$boldsymbol{x}$&lt;/tex-math&gt;&lt;/inline-formula&gt;. First, we consider a &lt;inline-formula&gt;&lt;tex-math&gt;$k$&lt;/tex-math&gt;&lt;/inline-formula&gt;-sparse &lt;inline-formula&gt;&lt;tex-math&gt;$boldsymbol{x}$&lt;/tex-math&gt;&lt;/inline-formula&gt; and introduce the thresholded Wirtinger flow (TWF) algorithm that does not require the sparsity level &lt;inline-formula&gt;&lt;tex-math&gt;$k$&lt;/tex-math&gt;&lt;/inline-formula&gt;. TWF comprises two steps: the spectral initialization that identifies a point sufficiently close to &lt;inline-formula&gt;&lt;tex-math&gt;$boldsymbol{x}$&lt;/tex-math&gt;&lt;/inline-formula&gt; (up to a sign flip) when &lt;inline-formula&gt;&lt;tex-math&gt;$m=O(k^{2}log n)$&lt;/tex-math&gt;&lt;/inline-formula&gt;, and the thresholded gradient descent which, when provided a good initialization, produces a sequence linearly converging to &lt;inline-formula&gt;&lt;tex-math&gt;$boldsymbol{x}$&lt;/tex-math&gt;&lt;/inline-formula&gt; with &lt;inline-formula&gt;&lt;tex-math&gt;$m=O(klog n)$&lt;/tex-math&gt;&lt;/inline-formula&gt; measurements. Second, we explore the generative prior, assuming that &lt;inline-formula&gt;&lt;tex-math&gt;$boldsymbol{x}$&lt;/tex-math&gt;&lt;/inline-formula&gt; lies in the range of an &lt;inline-formula&gt;&lt;tex-math&gt;$L$&lt;/tex-math&gt;&lt;/inline-formula&gt;-Lipschitz continuous generative model with &lt;inline-formula&gt;&lt;tex-math&gt;$k$&lt;/tex-math&gt;&lt;/inline-formula&gt;-dimensional inputs in an &lt;inline-formula&gt;&lt;tex-math&gt;$ell_{2}$&lt;/tex-math&gt;&lt;/inline-formula&gt;-ball of radius &lt;inline-formula&gt;&lt;tex-math&gt;$r$&lt;/tex-math&gt;&lt;/inline-formula&gt;. With an estimate correlated with the signal, we develop the projected gradient descent (PGD) algorithm that also comprises two steps: the projected power method that provides an initial vector with &lt;inline-formula&gt;&lt;tex-math&gt;$Obig{(}sqrt{klog(L)/m}big{)}$&lt;/tex-math&gt;&lt;/inline-formula&gt; &lt;inline-formula&gt;&lt;tex-math&gt;$ell_{2}$&lt;/tex-math&gt;&lt;/inline-formula&gt;-error given &lt;inline-formula&gt;&lt;tex-math&gt;$m=O(klog(Lnr))$&lt;/tex-math&gt;&lt;/inline-formula&gt; measurements, and the projected gradient descent that refines the &lt;inline-formula&gt;&lt;tex-math&gt;$ell_{2}$&lt;/tex-math&gt;&lt;/inline-formula&gt;-error to &lt;inline-formula&gt;&lt;tex-math&gt;$O(delta)$&lt;/tex-math&gt;&lt;/inline-formula&gt; at a geometric rate when &lt;inline-formula&gt;&lt;tex-math&gt;$m=O(klogfrac{Lrn}{delta^{2}})$&lt;/tex-math&gt;&lt;/inline-formula&gt;. Experimental results corroborate our theoretical findings and show that: (i) our approach for the sparse case nota","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"477-492"},"PeriodicalIF":4.6,"publicationDate":"2025-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142961228","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Multi-Fidelity Bayesian Optimization With Across-Task Transferable Max-Value Entropy Search 基于跨任务可转移最大值熵搜索的多保真贝叶斯优化
IF 4.6 2区 工程技术
IEEE Transactions on Signal Processing Pub Date : 2025-01-10 DOI: 10.1109/TSP.2025.3528252
Yunchuan Zhang;Sangwoo Park;Osvaldo Simeone
{"title":"Multi-Fidelity Bayesian Optimization With Across-Task Transferable Max-Value Entropy Search","authors":"Yunchuan Zhang;Sangwoo Park;Osvaldo Simeone","doi":"10.1109/TSP.2025.3528252","DOIUrl":"10.1109/TSP.2025.3528252","url":null,"abstract":"In many applications, ranging from logistics to engineering, a designer is faced with a sequence of optimization tasks for which the objectives are in the form of black-box functions that are costly to evaluate. Furthermore, higher-fidelity evaluations of the optimization objectives often entail a larger cost. Existing multi-fidelity black-box optimization strategies select candidate solutions and fidelity levels with the goal of maximizing the information about the optimal value or the optimal solution for the current task. Assuming that successive optimization tasks are related, this paper introduces a novel information-theoretic acquisition function that balances the need to acquire information about the current task with the goal of collecting information transferable to future tasks. The proposed method transfers across tasks distributions over parameters of a Gaussian process surrogate model by implementing particle-based variational Bayesian updates. Theoretical insights based on the analysis of the expected regret substantiate the benefits of acquiring transferable knowledge across tasks. Furthermore, experimental results across synthetic and real-world examples reveal that the proposed acquisition strategy that caters to future tasks can significantly improve the optimization efficiency as soon as a sufficient number of tasks is processed.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"418-432"},"PeriodicalIF":4.6,"publicationDate":"2025-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142961197","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Adaptive Polyak Step-Size for Momentum Accelerated Stochastic Gradient Descent With General Convergence Guarantee 广义收敛保证动量加速随机梯度下降的自适应Polyak步长
IF 4.6 2区 工程技术
IEEE Transactions on Signal Processing Pub Date : 2025-01-10 DOI: 10.1109/TSP.2025.3528217
Jiawei Zhang;Cheng Jin;Yuantao Gu
{"title":"Adaptive Polyak Step-Size for Momentum Accelerated Stochastic Gradient Descent With General Convergence Guarantee","authors":"Jiawei Zhang;Cheng Jin;Yuantao Gu","doi":"10.1109/TSP.2025.3528217","DOIUrl":"10.1109/TSP.2025.3528217","url":null,"abstract":"Momentum accelerated stochastic gradient descent (SGDM) has gained significant popularity in several signal processing and machine learning tasks. Despite its widespread success, the step size of SGDM remains a critical hyperparameter affecting its performance and often requires manual tuning. Recently, some works have introduced the Polyak step size to SGDM and provided corresponding convergence analysis. However, the convergence guarantee of existing Polyak step sizes for SGDM are limited to convex objectives and lack theoretical support for more widely applicable non-convex problems. To bridge this gap, we design a novel Polyak adaptive step size for SGDM. The proposed algorithm, termed SGDM-APS, incorporates a moving average form tailored for the momentum mechanism in SGDM. We establish the convergence guarantees of SGDM-APS for both convex and non-convex objectives, providing theoretical analysis of its effectiveness. To the best of our knowledge, SGDM-APS is the first Polyak step size for SGDM with general convergence guarantee. Our analysis can also be extended to constant step size SGDM, enriching the theoretical comprehension of the classic SGDM algorithm. Through extensive experiments on diverse benchmarks, we demonstrate that SGDM-APS achieves competitive convergence rates and generalization performance compared to several popular optimization algorithms.","PeriodicalId":13330,"journal":{"name":"IEEE Transactions on Signal Processing","volume":"73 ","pages":"462-476"},"PeriodicalIF":4.6,"publicationDate":"2025-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142961606","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信