Hyperparameter Estimation for Sparse Bayesian Learning Models

SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 3, Page 759-787, September 2024.
Abstract.Sparse Bayesian learning (SBL) models are extensively used in signal processing and machine learning for promoting sparsity through hierarchical priors. The hyperparameters in SBL models are crucial for the model’s performance, but they are often difficult to estimate due to the nonconvexity and the high-dimensionality of the associated objective function. This paper presents a comprehensive framework for hyperparameter estimation in SBL models, encompassing well-known algorithms such as the expectation-maximization, MacKay, and convex bounding algorithms. These algorithms are cohesively interpreted within an alternating minimization and linearization (AML) paradigm, distinguished by their unique linearized surrogate functions. Additionally, a novel algorithm within the AML framework is introduced, showing enhanced efficiency, especially under low signal noise ratios. This is further improved by a new alternating minimization and quadratic approximation paradigm, which includes a proximal regularization term. The paper substantiates these advancements with thorough convergence analysis and numerical experiments, demonstrating the algorithm’s effectiveness in various noise conditions and signal-to-noise ratios.