{"title":"Some New Results on the Maximum Growth Factor in Gaussian Elimination","authors":"Alan Edelman, John Urschel","doi":"10.1137/23m1571903","DOIUrl":"https://doi.org/10.1137/23m1571903","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 2, Page 967-991, June 2024. <br/> Abstract. This paper combines modern numerical computation with theoretical results to improve our understanding of the growth factor problem for Gaussian elimination. On the computational side we obtain lower bounds for the maximum growth for complete pivoting for [math] and [math] using the Julia JuMP optimization package. At [math] we obtain a growth factor bigger than [math]. The numerical evidence suggests that the maximum growth factor is bigger than [math] if and only if [math]. We also present a number of theoretical results. We show that the maximum growth factor over matrices with entries restricted to a subset of the reals is nearly equal to the maximum growth factor over all real matrices. We also show that the growth factors under floating point arithmetic and exact arithmetic are nearly identical. Finally, through numerical search, and stability and extrapolation results, we provide improved lower bounds for the maximum growth factor. Specifically, we find that the largest growth factor is bigger than [math] for [math], and the lim sup of the ratio with [math] is greater than or equal to [math]. In contrast to the old conjecture that growth might never be bigger than [math], it seems likely that the maximum growth divided by [math] goes to infinity as [math].","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140800244","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Conditioning of Matrix Functions at Quasi-Triangular Matrices","authors":"Awad H. Al-Mohy","doi":"10.1137/22m1543689","DOIUrl":"https://doi.org/10.1137/22m1543689","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 2, Page 954-966, June 2024. <br/>Abstract. The area of matrix functions has received growing interest for a long period of time due to their growing applications. Having a numerical algorithm for a matrix function, the ideal situation is to have an estimate or bound for the error returned alongside the solution. Condition numbers serve this purpose; they measure the first-order sensitivity of matrix functions to perturbations in the input data. We have observed that the existing unstructured condition number leads most of the time to inferior bounds of relative forward errors for some matrix functions at triangular and quasi-triangular matrices. We propose a condition number of matrix functions exploiting the structure of triangular and quasi-triangular matrices. We then adapt an existing algorithm for exact computation of the unstructured condition number to an algorithm for exact evaluation of the structured condition number. Although these algorithms are direct rather than iterative and useful for testing the numerical stability of numerical algorithms, they are less practical for relatively large problems. Therefore, we use an implicit power method approach to estimate the structured condition number. Our numerical experiments show that the structured condition number captures the behavior of the numerical algorithms and provides sharp bounds for the relative forward errors. In addition, the experiment indicates that the power method algorithm is reliable to estimate the structured condition number.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-04-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140800267","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Maike Meier, Yuji Nakatsukasa, Alex Townsend, Marcus Webb
{"title":"Are Sketch-and-Precondition Least Squares Solvers Numerically Stable?","authors":"Maike Meier, Yuji Nakatsukasa, Alex Townsend, Marcus Webb","doi":"10.1137/23m1551973","DOIUrl":"https://doi.org/10.1137/23m1551973","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 2, Page 905-929, June 2024. <br/> Abstract. Sketch-and-precondition techniques are efficient and popular for solving large least squares (LS) problems of the form [math] with [math] and [math]. This is where [math] is “sketched” to a smaller matrix [math] with [math] for some constant [math] before an iterative LS solver computes the solution to [math] with a right preconditioner [math], where [math] is constructed from [math]. Prominent sketch-and-precondition LS solvers are Blendenpik and LSRN. We show that the sketch-and-precondition technique in its most commonly used form is not numerically stable for ill-conditioned LS problems. For provable and practical backward stability and optimal residuals, we suggest using an unpreconditioned iterative LS solver on [math] with [math]. Provided the condition number of [math] is smaller than the reciprocal of the unit roundoff, we show that this modification ensures that the computed solution has a backward error comparable to the iterative LS solver applied to a well-conditioned matrix. Using smoothed analysis, we model floating-point rounding errors to argue that our modification is expected to compute a backward stable solution even for arbitrarily ill-conditioned LS problems. Additionally, we provide experimental evidence that using the sketch-and-solve solution as a starting vector in sketch-and-precondition algorithms (as suggested by Rokhlin and Tygert in 2008) should be highly preferred over the zero vector. The initialization often results in much more accurate solutions—albeit not always backward stable ones.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-04-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140800378","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Permutation-Invariant Log-Euclidean Geometries on Full-Rank Correlation Matrices","authors":"Yann Thanwerdas","doi":"10.1137/22m1538144","DOIUrl":"https://doi.org/10.1137/22m1538144","url":null,"abstract":"","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-04-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140665519","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Block Householder–Based Algorithm for the QR Decomposition of Hierarchical Matrices","authors":"Vincent Griem, Sabine Le Borne","doi":"10.1137/22m1544555","DOIUrl":"https://doi.org/10.1137/22m1544555","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 2, Page 847-874, June 2024. <br/> Abstract. Hierarchical matrices are dense but data-sparse matrices that use low-rank factorizations of suitable submatrices to reduce the storage and computational cost to linear-polylogarithmic complexity. In this paper, we propose a new approach to efficiently compute QR factorizations in the hierarchical matrix format based on block Householder transformations. To prevent unnecessarily high ranks in the resulting factors and to increase speed and accuracy, the algorithm meticulously tracks for which intermediate results low-rank factorizations are available. We also use a special storage scheme for the block Householder reflector to further reduce computational and storage costs. Numerical tests for two- and three-dimensional Laplacian boundary element matrices, different radial basis function kernel matrices, and matrices of typical hierarchical matrix structures but filled with random entries illustrate the performance of the new algorithm in comparison to some other QR algorithms for hierarchical matrices from the literature.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-04-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140627286","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Stochastic [math]th Root Approximation of a Stochastic Matrix: A Riemannian Optimization Approach","authors":"Fabio Durastante, Beatrice Meini","doi":"10.1137/23m1589463","DOIUrl":"https://doi.org/10.1137/23m1589463","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 2, Page 875-904, June 2024. <br/> Abstract. We propose two approaches, based on Riemannian optimization for computing a stochastic approximation of the [math]th root of a stochastic matrix [math]. In the first approach, the approximation is found in the Riemannian manifold of positive stochastic matrices. In the second approach, we introduce the Riemannian manifold of positive stochastic matrices sharing with [math] the Perron eigenvector and we compute the approximation of the [math]th root of [math] in such a manifold. This way, differently from the available methods based on constrained optimization, [math] and its [math]th root approximation share the Perron eigenvector. Such a property is relevant, from a modeling point of view, in the embedding problem for Markov chains. The extended numerical experimentation shows that, in the first approach, the Riemannian optimization methods are generally faster and more accurate than the available methods based on constrained optimization. In the second approach, even though the stochastic approximation of the [math]th root is found in a smaller set, the approximation is generally more accurate than the one obtained by standard constrained optimization.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-04-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140631166","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Analyzing Vector Orthogonalization Algorithms","authors":"Christopher C. Paige","doi":"10.1137/22m1519523","DOIUrl":"https://doi.org/10.1137/22m1519523","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 2, Page 829-846, June 2024. <br/> Abstract. Computer implementations of vector orthogonalization algorithms produce a sequence of supposedly orthogonal vectors, but rounding-errors can cause loss of orthogonality and rank. Nevertheless these computational algorithms can be very effective as parts of various methods. We develop a general theory based on the augmented orthogonal matrix developed in [SIAM J. Matrix Anal. Appl., 31 (2009), pp. 565–583] that can be applied to any such algorithm. This can be combined with a rounding-error analysis of the algorithm to analyze its finite-precision behavior. We apply this combination to prove that a particular Lanczos tridiagonalization of a Hermitian matrix always computes components for which backward-stable solutions to [math], [math], exist. If an appropriate rounding-error analysis is available, the approach can apparently be applied to any computation producing a sequence of supposedly orthogonal [math]-vectors, where a linear combination of these vectors is intended to approximate some quantity.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140581516","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Explicit Quantum Circuits for Block Encodings of Certain Sparse Matrices","authors":"Daan Camps, Lin Lin, Roel Van Beeumen, Chao Yang","doi":"10.1137/22m1484298","DOIUrl":"https://doi.org/10.1137/22m1484298","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 801-827, March 2024. <br/> Abstract. Many standard linear algebra problems can be solved on a quantum computer by using recently developed quantum linear algebra algorithms that make use of block encodings and quantum eigenvalue/singular value transformations. A block encoding embeds a properly scaled matrix of interest [math] in a larger unitary transformation [math] that can be decomposed into a product of simpler unitaries and implemented efficiently on a quantum computer. Although quantum algorithms can potentially achieve exponential speedup in solving linear algebra problems compared to the best classical algorithm, such a gain in efficiency ultimately hinges on our ability to construct an efficient quantum circuit for the block encoding of [math], which is difficult in general, and not trivial even for well structured sparse matrices. In this paper, we give a few examples on how efficient quantum circuits can be explicitly constructed for some well structured sparse matrices and discuss a few strategies used in these constructions. We also provide implementations of these quantum circuits in MATLAB.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140106580","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Matthias Christandl, Fulvio Gesmundo, Vladimir Lysikov, Vincent Steffan
{"title":"Partial Degeneration of Tensors","authors":"Matthias Christandl, Fulvio Gesmundo, Vladimir Lysikov, Vincent Steffan","doi":"10.1137/23m1554898","DOIUrl":"https://doi.org/10.1137/23m1554898","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 771-800, March 2024. <br/> Abstract. Tensors are often studied by introducing preorders such as restriction and degeneration. The former describes transformations of the tensors by local linear maps on its tensor factors; the latter describes transformations where the local linear maps may vary along a curve, and the resulting tensor is expressed as a limit along this curve. In this work, we introduce and study partial degeneration, a special version of degeneration where one of the local linear maps is constant while the others vary along a curve. Motivated by algebraic complexity, quantum entanglement, and tensor networks, we present constructions based on matrix multiplication tensors and find examples by making a connection to the theory of prehomogeneous tensor spaces. We highlight the subtleties of this new notion by showing obstruction and classification results for the unit tensor. To this end, we study the notion of aided rank, a natural generalization of tensor rank. The existence of partial degenerations gives strong upper bounds on the aided rank of a tensor, which allows one to turn degenerations into restrictions. In particular, we present several examples, based on the W-tensor and the Coppersmith–Winograd tensors, where lower bounds on aided rank provide obstructions to the existence of certain partial degenerations.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-03-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140097723","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Adaptive Rational Krylov Methods for Exponential Runge–Kutta Integrators","authors":"Kai Bergermann, Martin Stoll","doi":"10.1137/23m1559439","DOIUrl":"https://doi.org/10.1137/23m1559439","url":null,"abstract":"SIAM Journal on Matrix Analysis and Applications, Volume 45, Issue 1, Page 744-770, March 2024. <br/> Abstract. We consider the solution of large stiff systems of ODEs with explicit exponential Runge–Kutta integrators. These problems arise from semidiscretized semilinear parabolic PDEs on continuous domains or on inherently discrete graph domains. A series of results reduces the requirement of computing linear combinations of [math]-functions in exponential integrators to the approximation of the action of a smaller number of matrix exponentials on certain vectors. State-of-the-art computational methods use polynomial Krylov subspaces of adaptive size for this task. They have the drawback that the required number of Krylov subspace iterations to obtain a desired tolerance increases drastically with the spectral radius of the discrete linear differential operator, e.g., the problem size. We present an approach that leverages rational Krylov subspace methods promising superior approximation qualities. We prove a novel a posteriori error estimate of rational Krylov approximations to the action of the matrix exponential on vectors for single time points, which allows for an adaptive approach similar to existing polynomial Krylov techniques. We discuss pole selection and the efficient solution of the arising sequences of shifted linear systems by direct and preconditioned iterative solvers. Numerical experiments show that our method outperforms the state of the art for sufficiently large spectral radii of the discrete linear differential operators. The key to this are approximately constant numbers of rational Krylov iterations, which enable a near-linear scaling of the runtime with respect to the problem size.","PeriodicalId":49538,"journal":{"name":"SIAM Journal on Matrix Analysis and Applications","volume":null,"pages":null},"PeriodicalIF":1.5,"publicationDate":"2024-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140035180","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}