Tuning-Free Online Robust Principal Component Analysis Through Implicit Regularization

Abstract

The performance of Online Robust Principal Component Analysis (OR-PCA) technique heavily depends on the optimum tuning of the explicit regularizers. This tuning is dataset-sensitive and often impractical to optimize in real-world scenarios. We aim to remove the dependency on these tuning parameters by using implicit regularization. To this end, we develop an approach that integrates implicit regularization properties of various gradient descent methods to estimate sparse outliers and low-dimensional representations in a streaming setting—a non-trivial extension of existing techniques. A key novelty lies in the design of a new parameterization for matrix estimation in OR-PCA. Our method incorporates three different versions of modified gradient descent that separate but naturally encourage sparsity and low-rank structures in the data. Experimental results on synthetic and real-world video datasets demonstrate that the proposed method, namely, Tuning-Free OR-PCA (TF-ORPCA), outperforms existing OR-PCA methods. TF-ORPCA makes it more scalable for large datasets.