{"title":"An analysis of the total least squares problem","authors":"G. Golub, C. Loan","doi":"10.1137/0717073","DOIUrl":"https://doi.org/10.1137/0717073","url":null,"abstract":"Totla least squares (TLS) is a method of fitting that is appropriate when there are errors in both the observation vector $b (mxl)$ and in the data matrix $A (mxn)$. The technique has been discussed by several authors and amounts to fitting a \"best\" subspace to the points $(a^{T}_{i},b_{i}), i=1,ldots,m,$ where $a^{T}_{i}$ is the $i$-th row of $A$. In this paper a singular value decomposition analysis of the TLS problem is presented. The sensitivity of the TLS problem as well as its relationship to ordinary least squares regression is explored. Aan algorithm for solving the TLS problem is proposed that utilizes the singular value decomposition and which provides a measure of the underlying problem''s sensitivity.","PeriodicalId":250823,"journal":{"name":"Milestones in Matrix Computation","volume":"242 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1980-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121320268","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Generalized cross-validation as a method for choosing a good ridge parameter","authors":"G. Golub, M. Heath, G. Wahba","doi":"10.1080/00401706.1979.10489751","DOIUrl":"https://doi.org/10.1080/00401706.1979.10489751","url":null,"abstract":"Consider the ridge estimate (λ) for β in the model unknown, (λ) = (X T X + nλI)−1 X T y. We study the method of generalized cross-validation (GCV) for choosing a good value for λ from the data. The estimate is the minimizer of V(λ) given by where A(λ) = X(X T X + nλI)−1 X T . This estimate is a rotation-invariant version of Allen's PRESS, or ordinary cross-validation. This estimate behaves like a risk improvement estimator, but does not require an estimate of σ2, so can be used when n − p is small, or even if p ≥ 2 n in certain cases. The GCV method can also be used in subset selection and singular value truncation methods for regression, and even to choose from among mixtures of these methods.","PeriodicalId":250823,"journal":{"name":"Milestones in Matrix Computation","volume":"163 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1979-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116354478","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Ill-conditioned eigensystems and the computation of the Jordan canonical form","authors":"G. Golub, J. H. Wilkinson","doi":"10.1137/1018113","DOIUrl":"https://doi.org/10.1137/1018113","url":null,"abstract":"The solution of the complete eigenvalue problem for a non-normal matrix A presents severe practical difficulties when A is defective or close to a defective matrix. However in the presence of rounding errors one cannot even determine whether or not a matrix is defective. Several of the more stable methods for computing the Jordan canonical form are discussed together with the alternative approach of computing well-defined bases (usually orthogonal) of the relevant invariant subspaces.","PeriodicalId":250823,"journal":{"name":"Milestones in Matrix Computation","volume":"29 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1975-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130665346","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Some modified matrix eigenvalue problems","authors":"G. Golub","doi":"10.1137/1015032","DOIUrl":"https://doi.org/10.1137/1015032","url":null,"abstract":"We consider the numerical calculation of several matrix eigenvalue problems which require some manipulation before the standard algorithms may be used. This includes finding the stationary values of a quadratic form subject to linear constraints and determining the eigenvalues of a matrix which is modified by a matrix of rank one. We also consider several inverse eigenvalue problems. This includes the problem of determining the coefficients for the Gauss–Radau and Gauss–Lobatto quadrature rules. In addition, we study several eigenvalue problems which arise in least squares.","PeriodicalId":250823,"journal":{"name":"Milestones in Matrix Computation","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1973-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129431231","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Methods for modifying matrix factorizations","authors":"G. Golub, P. Gill, W. Murray, M. Saunders","doi":"10.1090/S0025-5718-1974-0343558-6","DOIUrl":"https://doi.org/10.1090/S0025-5718-1974-0343558-6","url":null,"abstract":"In recent years several algorithms have appeared for modifying the factors of a matrix following a rank-one change. These methods have always been given in the context of specific applications and this has probably inhibited their use over a wider field. In this report several methods are described for modifying Cholesky factors. Some of these have been published previously while others appear for the first time. In addition, a new algorithm is presented for modifying the complete orthogonal factorization of a general matrix, from which the conventional QR factors are obtained as a special case. A uniform notation has been used and emphasis has been placed on illustrating the similarity between different methods.","PeriodicalId":250823,"journal":{"name":"Milestones in Matrix Computation","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1972-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133068061","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The differentiation of pseudo-inverses and non-linear least squares problems whose variables separate","authors":"G. Golub, V. Pereyra","doi":"10.1137/0710036","DOIUrl":"https://doi.org/10.1137/0710036","url":null,"abstract":"For given data ($t_i , y_i), i=1, ldots ,m$ , we consider the least squares fit of nonlinear models of the form F($underset ~to a , underset ~to alpha ; t) = sum_{j=1}^{n} g_j (underset ~to a ) varphi_j (underset ~to alpha ; t) , underset ~to a epsilon R^s , underset ~to alpha epsilon R^k $. For this purpose we study the minimization of the nonlinear functional r($underset ~to a , underset ~to alpha ) = sum_{i=1}^{m} {(y_i - F(underset ~to a , underset ~to alpha , t_i))}^2$. It is shown that by defining the matrix ${ {Phi (underset ~to alpha} }_{i,j} = varphi_j (underset ~to alpha ; t_i)$ , and the modified functional $r_2(underset ~to alpha ) = l underset ~to y - Phi (underset ~to alpha )Phi^+(underset ~to alpha ) underset ~to y l_2^2$, it is possible to optimize first with respect to the parameters $underset ~to alpha$ , and then to obtain, a posteriori, the optimal parameters $overset ^to {underset ~to a}$. The matrix $Phi^+(underset ~to alpha$) is the Moore-Penrose generalized inverse of $Phi (underset ~to alpha$), and we develop formulas for its Frechet derivative under the hypothesis that $Phi (underset ~to alpha$) is of constant (though not necessarily full) rank. From these formulas we readily obtain the derivatives of the orthogonal projectors associated with $Phi (underset ~to alpha$), and also that of the functional $r_2(underset ~to alpha$). Detailed algorithms are presented which make extensive use of well-known reliable linear least squares techniques, and numerical results and comparisons are given. These results are generalizations of those of H. D. Scolnik [1971].","PeriodicalId":250823,"journal":{"name":"Milestones in Matrix Computation","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1972-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128970421","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Numerical methods for computing angles between linear subspaces","authors":"A. Bjoerck, G. Golub","doi":"10.1090/S0025-5718-1973-0348991-3","DOIUrl":"https://doi.org/10.1090/S0025-5718-1973-0348991-3","url":null,"abstract":"Assume that two subspaces F and G of unitary space are defined as the ranges (or nullspaces) of given rectangular matrices A and B. Accurate numerical methods are developed for computing the principal angles $theta_k (F,G)$ and orthogonal sets of principal vectors $u_k epsilon F$ and $v_k epsilon G$, k = 1,2,..., q = dim(G) $leq$ dim(F). An important application in statistics is computing the canonical correlations $sigma_k = cos theta_k$ between two sets of variates. A perturbation analysis shows that the condition number for $theta_k$ essentially is max($kappa (A),kappa (B)$), where $kappa$ denotes the condition number of a matrix. The algorithms are based on a preliminary QR-factorization of A and B (or $A^H$ and $B^H$), for which either the method of Householder transformations (HT) or the modified Gram-Schmidt method (MGS) is used. Then cos $theta_k$ and sin $theta_k$ are computed as the singular values of certain related matrices. Experimental results are given, which indicates that MGS gives $theta_k$ with equal precision and fewer arithmetic operations than HT. However, HT gives principal vectors, which are orthogonal to working accuracy, which is not in general true for MGS. Finally the case when A and/or B are rank deficient is discussed.","PeriodicalId":250823,"journal":{"name":"Milestones in Matrix Computation","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1971-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122192617","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"On direct methods for solving Poisson's equation","authors":"B. L. Buzbee, G. Golub, C. Nielson","doi":"10.1137/0707049","DOIUrl":"https://doi.org/10.1137/0707049","url":null,"abstract":"Some efficient and accurate direct methods are developed for solving certain elliptic partial difference equations over a rectangle with Dirichlet, Neumann or periodic boundary conditions. Generalizations to higher dimensions and to L-shaped regions are included.","PeriodicalId":250823,"journal":{"name":"Milestones in Matrix Computation","volume":"48 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1970-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129802044","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Calculation of Gauss quadrature rules","authors":"G. Golub, John H. Welsch","doi":"10.1090/S0025-5718-69-99647-1","DOIUrl":"https://doi.org/10.1090/S0025-5718-69-99647-1","url":null,"abstract":"Most numerical integration techniques consist of approximating the integrand by a polynomial in a region or regions and then integrating the polynomial exactly. Often a complicated integrand can be factored into a non-negative ''weight'' function and another function better approximated by a polynomial, thus $int_{a}^{b} g(t)dt = int_{a}^{b} omega (t)f(t)dt approx sum_{i=1}^{N} w_i f(t_i)$. Hopefully, the quadrature rule ${{w_j, t_j}}_{j=1}^{N}$ corresponding to the weight function $omega$(t) is available in tabulated form, but more likely it is not. We present here two algorithms for generating the Gaussian quadrature rule defined by the weight function when: a) the three term recurrence relation is known for the orthogonal polynomials generated by $omega$(t), and b) the moments of the weight function are known or can be calculated.","PeriodicalId":250823,"journal":{"name":"Milestones in Matrix Computation","volume":"75 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1967-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127344880","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A generalized conjugate gradient method for non-symmetric systems of linear equations","authors":"P. Concus, G. Golub","doi":"10.1007/978-3-642-85972-4_4","DOIUrl":"https://doi.org/10.1007/978-3-642-85972-4_4","url":null,"abstract":"","PeriodicalId":250823,"journal":{"name":"Milestones in Matrix Computation","volume":"68 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"1900-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116014753","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}