Numerical Linear Algebra with Applications最新文献_第7页

Blockwise acceleration of alternating least squares for canonical tensor decomposition 正则张量分解中交替最小二乘的块方向加速

IF 4.3 3区数学

Numerical Linear Algebra with Applications Pub Date : 2023-06-21 DOI: 10.1002/nla.2516

D. Evans, Nan Ye

{"title":"Blockwise acceleration of alternating least squares for canonical tensor decomposition","authors":"D. Evans, Nan Ye","doi":"10.1002/nla.2516","DOIUrl":"https://doi.org/10.1002/nla.2516","url":null,"abstract":"The canonical polyadic (CP) decomposition of tensors is one of the most important tensor decompositions. While the well‐known alternating least squares (ALS) algorithm is often considered the workhorse algorithm for computing the CP decomposition, it is known to suffer from slow convergence in many cases and various algorithms have been proposed to accelerate it. In this article, we propose a new accelerated ALS algorithm that accelerates ALS in a blockwise manner using a simple momentum‐based extrapolation technique and a random perturbation technique. Specifically, our algorithm updates one factor matrix (i.e., block) at a time, as in ALS, with each update consisting of a minimization step that directly reduces the reconstruction error, an extrapolation step that moves the factor matrix along the previous update direction, and a random perturbation step for breaking convergence bottlenecks. Our extrapolation strategy takes a simpler form than the state‐of‐the‐art extrapolation strategies and is easier to implement. Our algorithm has negligible computational overheads relative to ALS and is simple to apply. Empirically, our proposed algorithm shows strong performance as compared to the state‐of‐the‐art acceleration techniques on both simulated and real tensors.","PeriodicalId":49731,"journal":{"name":"Numerical Linear Algebra with Applications","volume":null,"pages":null},"PeriodicalIF":4.3,"publicationDate":"2023-06-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49113853","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The generalized residual cutting method and its convergence characteristics 广义残差切割方法及其收敛性

IF 4.3 3区数学

Numerical Linear Algebra with Applications Pub Date : 2023-06-20 DOI: 10.1002/nla.2517

T. Abe, Anthony T. Chronopoulos

引用次数: 0

A two‐step matrix splitting iteration paradigm based on one single splitting for solving systems of linear equations 求解线性方程组的基于一次分裂的两步矩阵分裂迭代范式

IF 4.3 3区数学

Numerical Linear Algebra with Applications Pub Date : 2023-06-11 DOI: 10.1002/nla.2510

Z. Bai

引用次数: 0

A Vanka‐based parameter‐robust multigrid relaxation for the Stokes–Darcy Brinkman problems Stokes-Darcy Brinkman问题的基于Vanka的参数鲁棒多网格松弛

3区数学

Numerical Linear Algebra with Applications Pub Date : 2023-06-05 DOI: 10.1002/nla.2514

Yunhui He

引用次数: 0

CP decomposition for tensors via alternating least squares with QR decomposition 用QR分解交替最小二乘对张量进行CP分解

IF 4.3 3区数学

Numerical Linear Algebra with Applications Pub Date : 2023-06-05 DOI: 10.1002/nla.2511

Rachel Minster, Irina Viviano, Xiaotian Liu, Grey Ballard

{"title":"CP decomposition for tensors via alternating least squares with QR decomposition","authors":"Rachel Minster, Irina Viviano, Xiaotian Liu, Grey Ballard","doi":"10.1002/nla.2511","DOIUrl":"https://doi.org/10.1002/nla.2511","url":null,"abstract":"The CP tensor decomposition is used in applications such as machine learning and signal processing to discover latent low‐rank structure in multidimensional data. Computing a CP decomposition via an alternating least squares (ALS) method reduces the problem to several linear least squares problems. The standard way to solve these linear least squares subproblems is to use the normal equations, which inherit special tensor structure that can be exploited for computational efficiency. However, the normal equations are sensitive to numerical ill‐conditioning, which can compromise the results of the decomposition. In this paper, we develop versions of the CP‐ALS algorithm using the QR decomposition and the singular value decomposition, which are more numerically stable than the normal equations, to solve the linear least squares problems. Our algorithms utilize the tensor structure of the CP‐ALS subproblems efficiently, have the same complexity as the standard CP‐ALS algorithm when the input is dense and the rank is small, and are shown via examples to produce more stable results when ill‐conditioning is present. Our MATLAB implementation achieves the same running time as the standard algorithm for small ranks, and we show that the new methods can obtain lower approximation error.","PeriodicalId":49731,"journal":{"name":"Numerical Linear Algebra with Applications","volume":null,"pages":null},"PeriodicalIF":4.3,"publicationDate":"2023-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48192707","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 1

Convergence acceleration of preconditioned conjugate gradient solver based on error vector sampling for a sequence of linear systems 基于误差向量采样的线性系统预处理共轭梯度求解器的收敛加速

IF 4.3 3区数学

Numerical Linear Algebra with Applications Pub Date : 2023-05-31 DOI: 10.1002/nla.2512

Takeshi Iwashita, Kota Ikehara, Takeshi Fukaya, T. Mifune

{"title":"Convergence acceleration of preconditioned conjugate gradient solver based on error vector sampling for a sequence of linear systems","authors":"Takeshi Iwashita, Kota Ikehara, Takeshi Fukaya, T. Mifune","doi":"10.1002/nla.2512","DOIUrl":"https://doi.org/10.1002/nla.2512","url":null,"abstract":"In this article, we focus on solving a sequence of linear systems that have identical (or similar) coefficient matrices. For this type of problem, we investigate subspace correction (SC) and deflation methods, which use an auxiliary matrix (subspace) to accelerate the convergence of the iterative method. In practical simulations, these acceleration methods typically work well when the range of the auxiliary matrix contains eigenspaces corresponding to small eigenvalues of the coefficient matrix. We develop a new algebraic auxiliary matrix construction method based on error vector sampling in which eigenvectors with small eigenvalues are efficiently identified in the solution process. We use the generated auxiliary matrix for convergence acceleration in the following solution step. Numerical tests confirm that both SC and deflation methods with the auxiliary matrix can accelerate the solution process of the iterative solver. Furthermore, we examine the applicability of our technique to the estimation of the condition number of the coefficient matrix. We also present the algorithm of the preconditioned conjugate gradient method with condition number estimation.","PeriodicalId":49731,"journal":{"name":"Numerical Linear Algebra with Applications","volume":null,"pages":null},"PeriodicalIF":4.3,"publicationDate":"2023-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44571147","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multilevel‐in‐width training for deep neural network regression 深度神经网络回归的多层次宽度训练

IF 4.3 3区数学

Numerical Linear Algebra with Applications Pub Date : 2023-05-19 DOI: 10.1002/nla.2501

Colin Ponce, Ruipeng Li, Christina Mao, P. Vassilevski

{"title":"Multilevel‐in‐width training for deep neural network regression","authors":"Colin Ponce, Ruipeng Li, Christina Mao, P. Vassilevski","doi":"10.1002/nla.2501","DOIUrl":"https://doi.org/10.1002/nla.2501","url":null,"abstract":"A common challenge in regression is that for many problems, the degrees of freedom required for a high‐quality solution also allows for overfitting. Regularization is a class of strategies that seek to restrict the range of possible solutions so as to discourage overfitting while still enabling good solutions, and different regularization strategies impose different types of restrictions. In this paper, we present a multilevel regularization strategy that constructs and trains a hierarchy of neural networks, each of which has layers that are wider versions of the previous network's layers. We draw intuition and techniques from the field of Algebraic Multigrid (AMG), traditionally used for solving linear and nonlinear systems of equations, and specifically adapt the Full Approximation Scheme (FAS) for nonlinear systems of equations to the problem of deep learning. Training through V‐cycles then encourage the neural networks to build a hierarchical understanding of the problem. We refer to this approach as multilevel‐in‐width to distinguish from prior multilevel works which hierarchically alter the depth of neural networks. The resulting approach is a highly flexible framework that can be applied to a variety of layer types, which we demonstrate with both fully connected and convolutional layers. We experimentally show with PDE regression problems that our multilevel training approach is an effective regularizer, improving the generalize performance of the neural networks studied.","PeriodicalId":49731,"journal":{"name":"Numerical Linear Algebra with Applications","volume":null,"pages":null},"PeriodicalIF":4.3,"publicationDate":"2023-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42963576","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Preconditioned tensor format conjugate gradient squared and biconjugate gradient stabilized methods for solving stein tensor equations 预条件张量格式求解stein张量方程的共轭梯度平方和双共轭梯度稳定方法

IF 4.3 3区数学

Numerical Linear Algebra with Applications Pub Date : 2023-05-10 DOI: 10.1002/nla.2502

Yuhan Chen, Chenliang Li

引用次数: 0

Issue Information 问题信息

IF 4.3 3区数学

Numerical Linear Algebra with Applications Pub Date : 2023-04-02 DOI: 10.1002/nla.2451

引用次数: 0

Anderson accelerated fixed‐point iteration for multilinear PageRank 多线性PageRank的Anderson加速不动点迭代

IF 4.3 3区数学

Numerical Linear Algebra with Applications Pub Date : 2023-03-28 DOI: 10.1002/nla.2499

Fuqi Lai, Wen Li, Xiaofei Peng, Yannan Chen

引用次数: 0