SIAM Review最新文献

{"title":"Combinatorial and Hodge Laplacians: Similarities and Differences","authors":"Emily Ribando-Gros, Rui Wang, Jiahui Chen, Yiying Tong, Guo-Wei Wei","doi":"10.1137/22m1482299","DOIUrl":"https://doi.org/10.1137/22m1482299","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 575-601, May 2024. <br/> As key subjects in spectral geometry and combinatorial graph theory, respectively, the (continuous) Hodge Laplacian and the combinatorial Laplacian share similarities in revealing the topological dimension and geometric shape of data and in their realization of diffusion and minimization of harmonic measures. It is believed that they also both associate with vector calculus, through the gradient, curl, and divergence, as argued in the popular usage of “Hodge Laplacians on graphs” in the literature. Nevertheless, these Laplacians are intrinsically different in their domains of definitions and applicability to specific data formats, hindering any in-depth comparison of the two approaches. For example, the spectral decomposition of a vector field on a simple point cloud using combinatorial Laplacians defined on some commonly used simplicial complexes does not give rise to the same curl-free and divergence-free components that one would obtain from the spectral decomposition of a vector field using either the continuous Hodge Laplacians defined on differential forms in manifolds or the discretized Hodge Laplacians defined on a point cloud with boundary in the Eulerian representation or on a regular mesh in the Eulerian representation. To facilitate the comparison and bridge the gap between the combinatorial Laplacian and Hodge Laplacian for the discretization of continuous manifolds with boundary, we further introduce boundary-induced graph (BIG) Laplacians using tools from discrete exterior calculus (DEC). BIG Laplacians are defined on discrete domains with appropriate boundary conditions to characterize the topology and shape of data. The similarities and differences among the combinatorial Laplacian, BIG Laplacian, and Hodge Laplacian are then examined. Through an Eulerian representation of 3D domains as level-set functions on regular grids, we show experimentally the conditions for the convergence of BIG Laplacian eigenvalues to those of the Hodge Laplacian for elementary shapes.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"367 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141904538","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Survey and Review","authors":"Marlis Hochbruck","doi":"10.1137/24n97592x","DOIUrl":"https://doi.org/10.1137/24n97592x","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 401-401, May 2024. <br/> In “Cardinality Minimization, Constraints, and Regularization: A Survey,\" Andreas M. Tillmann, Daniel Bienstock, Andrea Lodi, and Alexandra Schwartz consider a class of optimization problems that involve the cardinality of variable vectors in constraints or in the objective function. Such problems have many important applications, e.g., medical imaging (like X-ray tomography), face recognition, wireless sensor network design, stock picking, crystallography, astronomy, computer vision, classification and regression, interpretable machine learning, and statistical data analysis. The emphasis in this paper is on continuous variables, which distinguishes it from a myriad of classical operation research or combinatorial optimization problems. Three general problem classes are studied in detail: cardinality minimization problems, cardinality-constrained problems, and regularized cardinality problems. The paper provides a road map connecting several disciplines and offers an overview of many different computational approaches that are available for cardinality optimization problems. Since such problems are of cross-disciplinary nature, the authors organized their review according to specific application areas and point out overlaps and differences. The paper starts with prominent cardinality optimization problems, namely, signal and image processing, portfolio optimization and management, high-dimensional statistics and machine learning, and some related problems from combinatorics, matrix sparsification, and group/block sparsity. It then continues with exact models and solution methods. The further sections are devoted to relaxations and heuristics, scalability of exact and heuristic algorithms. The authors made a strong effort regarding the organization of their quite long paper, meaning that tables and figures guide the reader to an application or result of interest. In addition, they provide an extensive overview on the literature with more than 400 references.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"367 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141904644","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Cardinality Minimization, Constraints, and Regularization: A Survey","authors":"Andreas M. Tillmann, Daniel Bienstock, Andrea Lodi, Alexandra Schwartz","doi":"10.1137/21m142770x","DOIUrl":"https://doi.org/10.1137/21m142770x","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 403-477, May 2024. <br/> We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unified viewpoint on the general problem classes and models, and we give concrete examples from diverse application fields such as signal and image processing, portfolio selection, and machine learning. The paper discusses general-purpose modeling techniques and broadly applicable as well as problem-specific exact and heuristic solution approaches. While our perspective is that of mathematical optimization, a main goal of this work is to reach out to and build bridges between the different communities in which cardinality optimization problems are frequently encountered. In particular, we highlight that modern mixed-integer programming, which is often regarded as impractical due to the commonly unsatisfactory behavior of black-box solvers applied to generic problem formulations, can in fact produce provably high-quality or even optimal solutions for cardinality optimization problems, even in large-scale real-world settings. Achieving such performance typically involves drawing on the merits of problem-specific knowledge that may stem from different fields of application and, e.g., can shed light on structural properties of a model or its solutions, or can lead to the development of efficient heuristics. We also provide some illustrative examples.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"30 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141904540","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"SIGEST","authors":"The Editors","doi":"10.1137/24n975943","DOIUrl":"https://doi.org/10.1137/24n975943","url":null,"abstract":"SIAM Review, Volume 66, Issue 3, Page 533-533, May 2024. <br/> The SIGEST article in this issue is “Operator Learning Using Random Features: A Tool for Scientific Computing,” by Nicholas H. Nelsen and Andrew M. Stuart. This work considers the problem of operator learning in infinite-dimensional Banach spaces through the use of random features. The driving application is the approximation of solution operators to partial differential equations (PDEs), here foremost time-dependent problems, that are naturally posed in an infinite-dimensional function space. Typically here, in contrast to the mainstream big data regimes of machine learning applications such as computer vision, high resolution data coming from physical experiments or from computationally expensive simulations of such differential equations is usually small. Fast and approximate surrogates built from such data can be advantageous in building forward models for inverse problems or for doing uncertainty quantification, for instance. Showing how this can be done in infinite dimensions gives rise to approximators which are at the outset resolution and discretization invariant, allowing training on one resolution and deploying on another. At the heart of this work is the function-valued random features methodology that the authors extended from the finite setting of the classical random features approach. Here, the nonlinear operator is approximated by a linear combination of random operators which turn out to be a low-rank approximation and whose computation amounts to a convex, quadratic optimisation problem that is efficiently solvable and for which convergence guarantees can be derived. The methodology is then concretely applied to two concrete PDE examples: Burgers' equations and Darcy flow, demonstrating the applicability of the function-valued random features method, its scalability, discretization invariance, and transferability. The original 2021 article, which appeared in SIAM's Journal on Scientific Computing, has attracted considerable attention. In preparing this SIGEST version, the authors have made numerous modifications and revisions. These include expanding the introductory section and the concluding remarks, condensing the technical content and making it more accessible, and adding a link to an open access GitHub repository that contains all data and code used to produce the results in the paper.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"64 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141904646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Survey and Review","authors":"Marlis Hochbruck","doi":"10.1137/24n975876","DOIUrl":"https://doi.org/10.1137/24n975876","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 203-203, May 2024. <br/> Inverse problems arise in various applications---for instance, in geoscience, biomedical science, or mining engineering, to mention just a few. The purpose is to recover an object or phenomenon from measured data which is typically subject to noise. The article “Computational Methods for Large-Scale Inverse Problems: A Survey on Hybrid Projection Methods,” by Julianne Chung and Silvia Gazzola, focuses on large, mainly linear, inverse problems. The mathematical modeling of such problems results in a linear system with a very large matrix $A in mathbb{R}^{mtimes n}$ and a perturbed right-hand side. In some applications, it is not even possible to store the matrix, and thus algorithms which only use $A$ in the form of matrix-vector products $Ax$ or $A^Tx$ are the only choice. The article starts with two examples from image deblurring and tomographic reconstruction illustrating the challenges of inverse problems. It then presents the basic idea of regularization which consists of augmenting the model by additional information. Two variants of regularization methods are considered in detail, namely, variational and iterative methods. For variational methods it is crucial to know a good regularization parameter in advance. Unfortunately, its estimation can be expensive. On the other hand, iterative schemes, such as Krylov subspace methods, regularize by early termination of the iterations. Hybrid methods combine these two approaches leveraging the best features of each class. The paper focuses on hybrid projection methods. Here, one starts with a Krylov process in which the original problem is projected onto a low-dimensional subspace. The projected problem is then solved using a variational regularization method. The paper reviews the most relevant direct and iterative regularization techniques before it provides details on the two main building blocks of hybrid methods, namely, generating a subspace for the solution and solving the projected problem. It covers theoretical as well as numerical aspects of these schemes and also presents some extensions of hybrid methods: more general Tikhonov problems, nonstandard projection methods (enrichment, augmentation, recycling), $ell_p$ regularization, Bayesian setting, and nonlinear problems. In addition, relevant software packages are provided. The presentation is very clear and the paper is also readable for those who are not experts in the field. Hence, it is valuable for everyone interested in large-scale inverse problems.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"17 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140902936","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Dynamics of Signaling Games","authors":"Hannelore De Silva, Karl Sigmund","doi":"10.1137/23m156402x","DOIUrl":"https://doi.org/10.1137/23m156402x","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 368-387, May 2024. <br/> This tutorial describes several basic and much-studied types of interactions with incomplete information, analyzing them by means of evolutionary game dynamics. The games include sender-receiver games, owner-challenger contests, costly advertising, and calls for help. We model the evolution of populations of players reacting to each other and compare adaptive dynamics, replicator dynamics, and best-reply dynamics. In particular, we study signaling norms and nonequilibrium outcomes.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"32 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140903002","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Computational Methods for Large-Scale Inverse Problems: A Survey on Hybrid Projection Methods","authors":"Julianne Chung, Silvia Gazzola","doi":"10.1137/21m1441420","DOIUrl":"https://doi.org/10.1137/21m1441420","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 205-284, May 2024. <br/> This paper surveys an important class of methods that combine iterative projection methods and variational regularization methods for large-scale inverse problems. Iterative methods such as Krylov subspace methods are invaluable in the numerical linear algebra community and have proved important in solving inverse problems due to their inherent regularizing properties and their ability to handle large-scale problems. Variational regularization describes a broad and important class of methods that are used to obtain reliable solutions to inverse problems, whereby one solves a modified problem that incorporates prior knowledge. Hybrid projection methods combine iterative projection methods with variational regularization techniques in a synergistic way, providing researchers with a powerful computational framework for solving very large inverse problems. Although the idea of a hybrid Krylov method for linear inverse problems goes back to the 1980s, several recent advances on new regularization frameworks and methodologies have made this field ripe for extensions, further analyses, and new applications. In this paper, we provide a practical and accessible introduction to hybrid projection methods in the context of solving large (linear) inverse problems.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"24 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140903028","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Nonsmooth Optimization over the Stiefel Manifold and Beyond: Proximal Gradient Method and Recent Variants","authors":"Shixiang Chen, Shiqian Ma, Anthony Man-Cho So, Tong Zhang","doi":"10.1137/24m1628578","DOIUrl":"https://doi.org/10.1137/24m1628578","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 319-352, May 2024. <br/> We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving this class of problems converge slowly in practice, involve subproblems that can be as difficult as the original problem, or lack rigorous convergence guarantees. In this paper, we propose a manifold proximal gradient method (ManPG) for solving this class of problems. We prove that the proposed method converges globally to a stationary point and establish its iteration complexity for obtaining an $epsilon$-stationary point. Furthermore, we present numerical results on the sparse PCA and compressed modes problems to demonstrate the advantages of the proposed method. We also discuss some recent advances related to ManPG for Riemannian optimization with nonsmooth objective functions.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"105 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140902974","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"SIGEST","authors":"The Editors","doi":"10.1137/24n97589x","DOIUrl":"https://doi.org/10.1137/24n97589x","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 317-317, May 2024. <br/> The SIGEST article in this issue is “Nonsmooth Optimization over the Stiefel Manifold and Beyond: Proximal Gradient Method and Recent Variants,” by Shixiang Chen, Shiqian Ma, Anthony Man-Cho So, and Tong Zhang. This work considers nonsmooth optimization on the Stiefel manifold, the manifold of orthonormal $k$-frames in $mathbb{R}^n$. The authors propose a novel proximal gradient algorithm, coined ManPG, for minimizing the sum of a smooth, potentially nonconvex function, and a convex and potentially nonsmooth function whose arguments live on the Stiefel manifold. In contrast to existing approaches, which either are computationally expensive (due to expensive subproblems or slow convergence) or lack rigorous convergence guarantees, ManPG is thoroughly analyzed and features subproblems that can be computed efficiently. Nonsmooth optimization problems on the Stiefel manifold appear in many applications. In statistics sparse principal component analysis (PCA), that is, PCA that seeks principal components with very few nonzero entries, is a prime example. Unsupervised feature selection (machine learning) and blind deconvolution with a sparsity constraint on the deconvolved signal (inverse problems) are important instances of this general objective structure. At the heart of this work is a beautiful interplay between a theoretically well-founded and efficient novel optimization approach for an important class of problems and a set of computational experiments that demonstrate the effectiveness of this new approach. In order to make proximal gradient work for the Stiefel manifold they add a retraction step to the iterations that keeps the iterates feasible. The authors prove global convergence of ManPG to a stationary point and analyze its computational complexity for approximating the latter to $epsilon$ accuracy. The numerical discussion features results for sparse PCA and the problem of computing compressed modes, that is, spatially localized solutions, of the independent-particle Schrödinger equation. The original 2020 article, which appeared in SIAM Journal on Optimization, has attracted considerable attention. In preparing this SIGEST version, the authors have added a discussion on several subsequent works on algorithms for solving Riemannian optimization with nonsmooth objectives. These works were mostly motivated by the ManPG algorithm and include a manifold proximal point algorithm, manifold proximal linear algorithm, stochastic ManPG, zeroth-order ManPG, Riemannian proximal gradient method, and Riemannian proximal Newton method.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"20 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140903048","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A New Version of the Adaptive Fast Gauss Transform for Discrete and Continuous Sources","authors":"Leslie F. Greengard, Shidong Jiang, Manas Rachh, Jun Wang","doi":"10.1137/23m1572453","DOIUrl":"https://doi.org/10.1137/23m1572453","url":null,"abstract":"SIAM Review, Volume 66, Issue 2, Page 287-315, May 2024. <br/> We present a new version of the fast Gauss transform (FGT) for discrete and continuous sources. Classical Hermite expansions are avoided entirely, making use only of the plane-wave representation of the Gaussian kernel and a new hierarchical merging scheme. For continuous source distributions sampled on adaptive tensor product grids, we exploit the separable structure of the Gaussian kernel to accelerate the computation. For discrete sources, the scheme relies on the nonuniform fast Fourier transform (NUFFT) to construct near field plane-wave representations. The scheme has been implemented for either free-space or periodic boundary conditions. In many regimes, the speed is comparable to or better than that of the conventional FFT in work per grid point, despite being fully adaptive.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"27 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2024-05-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140902918","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}