Optimization methods for tensor decomposition: A comparison of new algorithms for fitting the CP(CANDECOMP/PARAFAC) model

Abstract

Tensor decomposition is widely used for multi-way data analysis and computations in chemical science. CP decomposition is one of the most useful tensor decomposition models for capturing the essential information in massive multi-way chemical data and for efficiently performing computations with such tensors. However, efficiently and accurately computing the tensor decomposition itself is a nontrivial problem that sometimes limits the advantage of tensor decomposition methods. In this work we propose and test three new decomposition algorithms, that are defined from extrapolation ideas applied to the alternating least square (ALS) algorithm for CP tensor decomposition. The performance of the proposed algorithms are validated on both a variety of simulated datasets and real experimental datasets including fluorescence spectroscopy data, hyperspectral data and electroencephalogram (EEG) data. The results show that the proposed algorithms significantly accelerate the standard CP-ALS decomposition while maintaining favorable accuracy. One of the proposed methods, denoted direct inversion of the iterative subspace-like extrapolated ALS(CP-AD), is inspired from widely used extrapolation procedures used in the context of solving non-linear equations in quantum chemistry, and shows a particular attractive combination of a much reduced number of iterations needed for convergence, and modest computational cost. For example, CP-AD provided resulting tensors of similar accuracy but significantly lower computational cost than the standard CP-ALS algorithm and the widely used line-search based CP-ALS extrapolation procedure. The proposed methodology may thereby boost the application of tensor decomposition modeling in both experimental and computational chemistry.