Steffen Dereich, Arnulf Jentzen, Sebastian Kassing
{"title":"On the Existence of Minimizers in Shallow Residual ReLU Neural Network Optimization Landscapes","authors":"Steffen Dereich, Arnulf Jentzen, Sebastian Kassing","doi":"10.1137/23m1556241","DOIUrl":"https://doi.org/10.1137/23m1556241","url":null,"abstract":"SIAM Journal on Numerical Analysis, Volume 62, Issue 6, Page 2640-2666, December 2024. <br/> Abstract. In this article, we show the existence of minimizers in the loss landscape for residual artificial neural networks (ANNs) with a multidimensional input layer and one hidden layer with ReLU activation. Our work contrasts with earlier results in [D. Gallon, A. Jentzen, and F. Lindner, preprint, arXiv:2211.15641, 2022] and [P. Petersen, M. Raslan, and F. Voigtlaender, Found. Comput. Math., 21 (2021), pp. 375–444] which showed that in many situations minimizers do not exist for common smooth activation functions even in the case where the target functions are polynomials. The proof of the existence property makes use of a closure of the search space containing all functions generated by ANNs and additional discontinuous generalized responses. As we will show, the additional generalized responses in this larger space are suboptimal so that the minimum is attained in the original function class.","PeriodicalId":49527,"journal":{"name":"SIAM Journal on Numerical Analysis","volume":"182 1","pages":""},"PeriodicalIF":2.9,"publicationDate":"2024-11-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142718615","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Domain Decomposition Method for Stochastic Evolution Equations","authors":"Evelyn Buckwar, Ana Djurdjevac, Monika Eisenmann","doi":"10.1137/24m1629845","DOIUrl":"https://doi.org/10.1137/24m1629845","url":null,"abstract":"SIAM Journal on Numerical Analysis, Volume 62, Issue 6, Page 2611-2639, December 2024. <br/> Abstract. In recent years, stochastic partial differential equations (SPDEs) have become a well-studied field in mathematics. With their increase in popularity, it becomes important to efficiently approximate their solutions. Thus, our goal is a contribution towards the development of efficient and practical time-stepping methods for SPDEs. Operator splitting schemes provide powerful, efficient, and flexible numerical methods for deterministic and stochastic differential equations. An example is given by domain decomposition schemes, where one splits the domain into subdomains and constructs the numerical approximation in a divide-and-conquer strategy. Instead of solving one expensive problem on the entire domain, one then deals with cheaper problems on the subdomains. This is particularly useful in modern computer architectures, as the subproblems may often be solved in parallel. While splitting methods have already been used to study domain decomposition methods for deterministic PDEs, this is a new approach for SPDEs. This implies that the existing convergence analysis is not directly applicable, even though the building blocks of the operator splitting domain decomposition method are standard. We provide an abstract convergence analysis of a splitting scheme for stochastic evolution equations and state a domain decomposition scheme as an application of the setting. The theoretical results are verified through numerical experiments.","PeriodicalId":49527,"journal":{"name":"SIAM Journal on Numerical Analysis","volume":"81 1","pages":""},"PeriodicalIF":2.9,"publicationDate":"2024-11-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142679164","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"New Time Domain Decomposition Methods for Parabolic Optimal Control Problems II: Neumann–Neumann Algorithms","authors":"Martin J. Gander, Liu-Di Lu","doi":"10.1137/24m1634424","DOIUrl":"https://doi.org/10.1137/24m1634424","url":null,"abstract":"SIAM Journal on Numerical Analysis, Volume 62, Issue 6, Page 2588-2610, December 2024. <br/> Abstract. We propose to use Neumann–Neumann algorithms for the time parallel solution of unconstrained linear parabolic optimal control problems. We study nine variants, analyze their convergence behavior, and determine the optimal relaxation parameter for each. Our findings indicate that while the most intuitive Neumann–Neumann algorithms act as effective smoothers, there are more efficient Neumann–Neumann solvers available. We support our analysis with numerical experiments.","PeriodicalId":49527,"journal":{"name":"SIAM Journal on Numerical Analysis","volume":"6 1","pages":""},"PeriodicalIF":2.9,"publicationDate":"2024-11-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142679163","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
J. A. Carrillo, F. Hoffmann, A. M. Stuart, U. Vaes
{"title":"The Mean-Field Ensemble Kalman Filter: Near-Gaussian Setting","authors":"J. A. Carrillo, F. Hoffmann, A. M. Stuart, U. Vaes","doi":"10.1137/24m1628207","DOIUrl":"https://doi.org/10.1137/24m1628207","url":null,"abstract":"SIAM Journal on Numerical Analysis, Volume 62, Issue 6, Page 2549-2587, December 2024. <br/> Abstract. The ensemble Kalman filter is widely used in applications because, for high-dimensional filtering problems, it has a robustness that is not shared, for example, by the particle filter; in particular, it does not suffer from weight collapse. However, there is no theory which quantifies its accuracy as an approximation of the true filtering distribution, except in the Gaussian setting. To address this issue, we provide the first analysis of the accuracy of the ensemble Kalman filter beyond the Gaussian setting. We prove two types of results: The first type comprises a stability estimate controlling the error made by the ensemble Kalman filter in terms of the difference between the true filtering distribution and a nearby Gaussian, and the second type uses this stability result to show that, in a neighborhood of Gaussian problems, the ensemble Kalman filter makes a small error in comparison with the true filtering distribution. Our analysis is developed for the mean-field ensemble Kalman filter. We rewrite the update equations for this filter and for the true filtering distribution in terms of maps on probability measures. We introduce a weighted total variation metric to estimate the distance between the two filters, and we prove various stability estimates for the maps defining the evolution of the two filters in this metric. Using these stability estimates, we prove results of the first and second types in the weighted total variation metric. We also provide a generalization of these results to the Gaussian projected filter, which can be viewed as a mean-field description of the unscented Kalman filter.","PeriodicalId":49527,"journal":{"name":"SIAM Journal on Numerical Analysis","volume":"29 1","pages":""},"PeriodicalIF":2.9,"publicationDate":"2024-11-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142642576","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Lanczos Tau Framework for Time-Delay Systems: Padé Approximation and Collocation Revisited","authors":"Evert Provoost, Wim Michiels","doi":"10.1137/24m164611x","DOIUrl":"https://doi.org/10.1137/24m164611x","url":null,"abstract":"SIAM Journal on Numerical Analysis, Volume 62, Issue 6, Page 2529-2548, December 2024. <br/> Abstract. We reformulate the Lanczos tau method for the discretization of time-delay systems in terms of a pencil of operators, allowing for new insights into this approach. As a first main result, we show that, for the choice of a shifted Legendre basis, this method is equivalent to Padé approximation in the frequency domain. We illustrate that Lanczos tau methods straightforwardly give rise to sparse, self-nesting discretizations. Equivalence is also demonstrated with pseudospectral collocation, where the nonzero collocation points are chosen as the zeros of orthogonal polynomials. The importance of such a choice manifests itself in the approximation of the [math]-norm, where, under mild conditions, supergeometric convergence is observed and, for a special case, superconvergence is proved, both of which are significantly faster than the algebraic convergence reported in previous work.","PeriodicalId":49527,"journal":{"name":"SIAM Journal on Numerical Analysis","volume":"163 1","pages":""},"PeriodicalIF":2.9,"publicationDate":"2024-11-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142610476","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Spherical Designs for Approximations on Spherical Caps","authors":"Chao Li, Xiaojun Chen","doi":"10.1137/23m1555417","DOIUrl":"https://doi.org/10.1137/23m1555417","url":null,"abstract":"SIAM Journal on Numerical Analysis, Volume 62, Issue 6, Page 2506-2528, December 2024. <br/> Abstract. A spherical [math]-design is a set of points on the unit sphere, which provides an equal weight quadrature rule integrating exactly all spherical polynomials of degree at most [math] and has a sharp error bound for approximations on the sphere. This paper introduces a set of points called a spherical cap [math]-subdesign on a spherical cap [math] with center [math] and radius [math] induced by the spherical [math]-design. We show that the spherical cap [math]-subdesign provides an equal weight quadrature rule integrating exactly all zonal polynomials of degree at most [math] and all functions expanded by orthonormal functions on the spherical cap which are defined by shifted Legendre polynomials of degree at most [math]. We apply the spherical cap [math]-subdesign and the orthonormal basis functions on the spherical cap to non-polynomial approximation of continuous functions on the spherical cap and present theoretical approximation error bounds. We also apply spherical cap [math]-subdesigns to sparse signal recovery on the upper hemisphere, which is a spherical cap with [math]. Our theoretical and numerical results show that spherical cap [math]-subdesigns can provide a good approximation on spherical caps.","PeriodicalId":49527,"journal":{"name":"SIAM Journal on Numerical Analysis","volume":"153 1","pages":""},"PeriodicalIF":2.9,"publicationDate":"2024-11-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142598300","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An Operator Preconditioned Combined Field Integral Equation for Electromagnetic Scattering","authors":"Van Chien Le, Kristof Cools","doi":"10.1137/23m1581674","DOIUrl":"https://doi.org/10.1137/23m1581674","url":null,"abstract":"SIAM Journal on Numerical Analysis, Volume 62, Issue 6, Page 2484-2505, December 2024. <br/> Abstract. This paper aims to address two issues of integral equations for the scattering of time-harmonic electromagnetic waves by a perfect electric conductor with Lipschitz continuous boundary: ill-conditioned boundary element Galerkin discretization matrices on fine meshes and instability at spurious resonant frequencies. The remedy to ill-conditioned matrices is operator preconditioning, and resonant instability is eliminated by means of a combined field integral equation. Exterior traces of single and double layer potentials are complemented by their interior counterparts for a purely imaginary wave number. We derive the corresponding variational formulation in the natural trace space for electromagnetic fields and establish its well-posedness for all wave numbers. A Galerkin discretization scheme is employed using conforming edge boundary elements on dual meshes, which produces well-conditioned discrete linear systems of the variational formulation. Some numerical results are also provided to support the numerical analysis.","PeriodicalId":49527,"journal":{"name":"SIAM Journal on Numerical Analysis","volume":"63 1","pages":""},"PeriodicalIF":2.9,"publicationDate":"2024-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142594676","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An Energy-Based Discontinuous Galerkin Method for the Nonlinear Schrödinger Equation with Wave Operator","authors":"Kui Ren, Lu Zhang, Yin Zhou","doi":"10.1137/23m1597496","DOIUrl":"https://doi.org/10.1137/23m1597496","url":null,"abstract":"SIAM Journal on Numerical Analysis, Volume 62, Issue 6, Page 2459-2483, December 2024. <br/> Abstract. This work develops an energy-based discontinuous Galerkin (EDG) method for the nonlinear Schrödinger equation with the wave operator. The focus of the study is on the energy-conserving or energy-dissipating behavior of the method with some simple mesh-independent numerical fluxes we designed. We establish error estimates in the energy norm that require careful selection of a weak formulation for the auxiliary equation involving the time derivative of the displacement variable. A critical part of the convergence analysis is to establish the [math] error bounds for the time derivative of the approximation error in the displacement variable by using the equation that determines its mean value. Using a special weak formulation, we show that one can create a linear system for the time evolution of the unknowns even when dealing with nonlinear properties in the original problem. Numerical experiments were performed to demonstrate the optimal convergence of the scheme in the [math] norm. These experiments involved specific choices of numerical fluxes combined with specific choices of approximation spaces.","PeriodicalId":49527,"journal":{"name":"SIAM Journal on Numerical Analysis","volume":"10 1","pages":""},"PeriodicalIF":2.9,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142580294","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Annalisa Buffa, Ondine Chanon, Denise Grappein, Rafael Vázquez, Martin Vohralík
{"title":"An Equilibrated Flux A Posteriori Error Estimator for Defeaturing Problems","authors":"Annalisa Buffa, Ondine Chanon, Denise Grappein, Rafael Vázquez, Martin Vohralík","doi":"10.1137/23m1627195","DOIUrl":"https://doi.org/10.1137/23m1627195","url":null,"abstract":"SIAM Journal on Numerical Analysis, Volume 62, Issue 6, Page 2439-2458, December 2024. <br/> Abstract. An a posteriori error estimator based on an equilibrated flux reconstruction is proposed for defeaturing problems in the context of finite element discretizations. Defeaturing consists in the simplification of a geometry by removing features that are considered not relevant for the approximation of the solution of a given PDE. In this work, the focus is on a Poisson equation with Neumann boundary conditions on the feature boundary. The estimator accounts both for the so-called defeaturing error and for the numerical error committed by approximating the solution on the defeatured domain. Unlike other estimators that were previously proposed for defeaturing problems, the use of the equilibrated flux reconstruction allows us to obtain a sharp bound for the numerical component of the error. Furthermore, it does not require the evaluation of the normal trace of the numerical flux on the feature boundary: this makes the estimator well suited for finite element discretizations, in which the normal trace of the numerical flux is typically discontinuous across elements. The reliability of the estimator is proven and verified on several numerical examples. Its capability to identify the most relevant features is also shown, in anticipation of a future application to an adaptive strategy.","PeriodicalId":49527,"journal":{"name":"SIAM Journal on Numerical Analysis","volume":"41 1","pages":""},"PeriodicalIF":2.9,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142580295","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The A Posteriori Error Estimates of the FE Approximation of Defective Eigenvalues for Non-Self-Adjoint Eigenvalue Problems","authors":"Yidu Yang, Shixi Wang, Hai Bi","doi":"10.1137/23m162065x","DOIUrl":"https://doi.org/10.1137/23m162065x","url":null,"abstract":"SIAM Journal on Numerical Analysis, Volume 62, Issue 6, Page 2419-2438, December 2024. <br/> Abstract. In this paper, we study the a posteriori error estimates of the FEM for defective eigenvalues of non-self-adjoint eigenvalue problems. Using the spectral approximation theory, we establish the abstract a posteriori error formulas for the weighted average of approximate eigenvalues and approximate eigenspace. We then apply the formulas to the defective eigenvalues of elliptic interface problem, derive the a posteriori error estimators, and analyze their reliability and effectiveness. We also provide numerical examples to confirm our theoretical findings.","PeriodicalId":49527,"journal":{"name":"SIAM Journal on Numerical Analysis","volume":"8 1","pages":""},"PeriodicalIF":2.9,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142580296","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}