The differentiation of pseudo-inverses and non-linear least squares problems whose variables separate

Abstract

For given data ($t_i\ , y_i), i=1, \ldots ,m$ , we consider the least squares fit of nonlinear models of the form F($\underset ~\to a\ , \underset ~\to \alpha\ ; t) = \sum_{j=1}^{n}\ g_j (\underset ~\to a ) \varphi_j (\underset ~\to \alpha\ ; t) , \underset ~\to a\ \epsilon R^s\ , \underset ~\to \alpha\ \epsilon R^k\ $. For this purpose we study the minimization of the nonlinear functional r($\underset ~\to a\ , \underset ~\to \alpha ) = \sum_{i=1}^{m} {(y_i - F(\underset ~\to a , \underset ~\to \alpha , t_i))}^2$. It is shown that by defining the matrix ${ \{\Phi (\underset ~\to \alpha\} }_{i,j} = \varphi_j (\underset ~\to \alpha ; t_i)$ , and the modified functional $r_2(\underset ~\to \alpha ) = \l\ \underset ~\to y\ - \Phi (\underset ~\to \alpha )\Phi^+(\underset ~\to \alpha ) \underset ~\to y \l_2^2$, it is possible to optimize first with respect to the parameters $\underset ~\to \alpha$ , and then to obtain, a posteriori, the optimal parameters $\overset ^\to {\underset ~\to a}$. The matrix $\Phi^+(\underset ~\to \alpha$) is the Moore-Penrose generalized inverse of $\Phi (\underset ~\to \alpha$), and we develop formulas for its Frechet derivative under the hypothesis that $\Phi (\underset ~\to \alpha$) is of constant (though not necessarily full) rank. From these formulas we readily obtain the derivatives of the orthogonal projectors associated with $\Phi (\underset ~\to \alpha$), and also that of the functional $r_2(\underset ~\to \alpha$). Detailed algorithms are presented which make extensive use of well-known reliable linear least squares techniques, and numerical results and comparisons are given. These results are generalizations of those of H. D. Scolnik [1971].