Eigenvalue problems via the multiplicative weights update method

Oja's algorithm is a well known online algorithm studied mainly in the context of stochastic principal component analysis. We make a simple observation, yet to the best of our knowledge a novel one: when applied to any (not necessarily stochastic) sequence of real symmetric matrices which share common eigenvectors, the regret of Oja's algorithm could be directly analyzed using the celebrated multiplicative weights update method for the problem of prediction with expert advice. This leads to several applications: I. Novel analysis of a projected gradient ascent method for the symmetric eigenvalue problem, which differs from classical power method-style analyzes. II. Extension of the latter to a more general class of problems which consider minimizing a convex nonsmooth objective composed with a quadratic mapping over the unit sphere, under a common eigenvectors assumption. III. New insights to the open problem of online learning of eigenvectors using only linear runtime and memory.