Greedy rank updates combined with Riemannian descent methods for low-rank optimization

Abstract

We present a rank-adaptive optimization strategy for finding low-rank solutions of matrix optimization problems involving a quadratic objective function. The algorithm combines a greedy outer iteration that increases the rank and a smooth Riemannian algorithm that further optimizes the cost function on a fixed-rank manifold. While such a strategy is not especially novel, we show that it can be interpreted as a perturbed gradient descent algorithms or as a simple warm-starting strategy of a projected gradient algorithm on the variety of matrices of bounded rank. In addition, our numerical experiments show that the strategy is very efficient for recovering full rank but highly ill-conditioned matrices that have small numerical rank.