Overfitting Avoidance in Tensor Train Factorization and Completion: Prior Analysis and Inference

Abstract

Tensor train (TT) decomposition, a powerful tool for analyzing multidimensional data, exhibits superior performance in many machine learning tasks. However, existing methods for TT decomposition either suffer from noise overfitting, or require extensive fine-tuning of the balance between model complexity and representation accuracy. In this paper, a fully Bayesian treatment of TT decomposition is employed to avoid noise overfitting without parameter tuning. In particular, theoretical evidence is established for adopting a Gaussian-product-Gamma prior to induce sparsity on the slices of the TT cores. Furthermore, based on the proposed probabilistic model, an efficient learning algorithm is derived under the variational inference framework. Experiments on real-world data demonstrate the proposed algorithm performs better in image completion and image classification, compared to other existing TT decomposition algorithms.