Modified Regularization for High-dimensional Data Decomposition

Abstract

With the increased dimensionality of datasets, high-dimensional data decomposition models have become essential data analysis tools. However, the decomposition method usually suffers from the overfitting problem and, consequently, cannot achieve state-of-the-art performance. This motivates the introduction of various regularization terms. The commonly applied Ridge regression has limited applicability for the asperity dataset and reduces performance for sparse data, while the Lasso regression has higher efficiency in the sparse dataset. To address this challenge, we propose a modified regularization term designed by integrating both the Lasso and Ridge regressions. The different roles of these two regressions are analyzed. By adjusting the weights of the regression in the regularization term, the existing decomposition method can be applied to the dataset with different degrees of sparsity. The experiments show that the modified regularization term yields consistent improvement in the performance of existing benchmarks.