SIGEST

Abstract

SIAM Review, Volume 66, Issue 2, Page 317-317, May 2024.
The SIGEST article in this issue is “Nonsmooth Optimization over the Stiefel Manifold and Beyond: Proximal Gradient Method and Recent Variants,” by Shixiang Chen, Shiqian Ma, Anthony Man-Cho So, and Tong Zhang. This work considers nonsmooth optimization on the Stiefel manifold, the manifold of orthonormal $k$-frames in $\mathbb{R}^n$. The authors propose a novel proximal gradient algorithm, coined ManPG, for minimizing the sum of a smooth, potentially nonconvex function, and a convex and potentially nonsmooth function whose arguments live on the Stiefel manifold. In contrast to existing approaches, which either are computationally expensive (due to expensive subproblems or slow convergence) or lack rigorous convergence guarantees, ManPG is thoroughly analyzed and features subproblems that can be computed efficiently. Nonsmooth optimization problems on the Stiefel manifold appear in many applications. In statistics sparse principal component analysis (PCA), that is, PCA that seeks principal components with very few nonzero entries, is a prime example. Unsupervised feature selection (machine learning) and blind deconvolution with a sparsity constraint on the deconvolved signal (inverse problems) are important instances of this general objective structure. At the heart of this work is a beautiful interplay between a theoretically well-founded and efficient novel optimization approach for an important class of problems and a set of computational experiments that demonstrate the effectiveness of this new approach. In order to make proximal gradient work for the Stiefel manifold they add a retraction step to the iterations that keeps the iterates feasible. The authors prove global convergence of ManPG to a stationary point and analyze its computational complexity for approximating the latter to $\epsilon$ accuracy. The numerical discussion features results for sparse PCA and the problem of computing compressed modes, that is, spatially localized solutions, of the independent-particle Schrödinger equation. The original 2020 article, which appeared in SIAM Journal on Optimization, has attracted considerable attention. In preparing this SIGEST version, the authors have added a discussion on several subsequent works on algorithms for solving Riemannian optimization with nonsmooth objectives. These works were mostly motivated by the ManPG algorithm and include a manifold proximal point algorithm, manifold proximal linear algorithm, stochastic ManPG, zeroth-order ManPG, Riemannian proximal gradient method, and Riemannian proximal Newton method.