SIAM ReviewPub Date : 2026-08-06DOI: 10.1137/24m1651241
Huan Zhang, Yifan Chen, Eric Vanden-Eijnden, Benjamin Peherstorfer
{"title":"Sequential-in-Time Training of Nonlinear Parametrizations for Solving Time-Dependent Partial Differential Equations","authors":"Huan Zhang, Yifan Chen, Eric Vanden-Eijnden, Benjamin Peherstorfer","doi":"10.1137/24m1651241","DOIUrl":"https://doi.org/10.1137/24m1651241","url":null,"abstract":"SIAM Review, Volume 68, Issue 3, Page 579-608, September 2026. <br/> Abstract.Sequential-in-time methods solve a sequence of training problems to fit nonlinear parametrizations such as neural networks to approximate solution trajectories of partial differential equations over time. This work shows that sequential-in-time training methods can be understood broadly as either optimize-then-discretize (OtD) or discretize-then-optimize (DtO) schemes, which are well-known concepts in numerical analysis. The unifying perspective leads to novel stability and a posteriori error analysis results that provide insights into theoretical and numerical aspects that are inherent to either OtD or DtO schemes such as the tangent space collapse phenomenon, which is a form of overfitting. Additionally, the unified perspective facilitates the establishment of connections between variants of sequential-in-time training methods, which is demonstrated by identifying natural gradient descent methods on energy functionals as OtD schemes applied to the corresponding gradient flows.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"83 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2026-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148687820","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM ReviewPub Date : 2026-08-06DOI: 10.1137/26m1886638
Alexis Goujon, Sebastian Neumayer, Stanislas Ducotterd, Michael Unser
{"title":"Data-Driven Regularization with Weak Convexity for Robust Image Reconstruction","authors":"Alexis Goujon, Sebastian Neumayer, Stanislas Ducotterd, Michael Unser","doi":"10.1137/26m1886638","DOIUrl":"https://doi.org/10.1137/26m1886638","url":null,"abstract":"SIAM Review, Volume 68, Issue 3, Page 611-634, September 2026. <br/> Abstract.We describe a practical framework for data-driven regularization in image reconstruction. It combines model expressivity with the guarantees of variational methods. The approach is based on a weakly convex ridge regularizer, defined as the composition of a convolutional filter bank and pointwise potentials constrained to be weakly convex. These potentials are implemented as learnable splines with a strict control of their weak-convexity modulus. The resulting denoisers outperform classic convex regularization techniques as well as competitive benchmarks such as BM3D, while they still correspond to the minimization of a convex energy. The learned regularizers further extend to general inverse problems with provable convergence to critical points. Overall, this framework shows that a controlled relaxation of convexity enables the design of learnable priors that achieve strong empirical performance while preserving mathematical guarantees.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"57 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2026-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148687818","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Augmented Lagrangian Methods: Overview and Recent Advances","authors":"Kangkang Deng, Rui Wang, Zhenyuan Zhu, Junyu Zhang, Zaiwen Wen","doi":"10.1137/24m1706153","DOIUrl":"https://doi.org/10.1137/24m1706153","url":null,"abstract":"SIAM Review, Volume 68, Issue 3, Page 471-576, September 2026. <br/> Abstract.Large-scale constrained optimization is pivotal in modern scientific, engineering, and industrial computation, often involving complex systems with numerous variables and constraints. This paper provides a unified and comprehensive perspective on constructing augmented Lagrangian functions (based on the Hestenes–Powell–Rockafellar augmented Lagrangian) for various optimization problems, including nonlinear programming and convex and nonconvex composite programming. We present the augmented Lagrangian method (ALM), covering its theoretical foundations in both convex and nonconvex cases, and discuss several successful examples and applications. Recent advancements have extended the ALM’s capabilities to handle nonconvex constraints and ensure global convergence to first- and second-order stationary points. For nonsmooth convex problems, the ALM utilizes proximal operations, preserving desirable properties such as locally linear convergence rates. Furthermore, recent progress has refined the complexity analysis of the ALM and tackled challenging integer programming instances. This review aims to offer a thorough understanding of the ALM’s benefits and limitations, exploring different ALM variants designed to enhance convergence and computational performance. We also illustrate effective algorithms for ALM subproblems across different types of optimization problems and highlight practical implementations in several fields.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"24 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2026-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148675337","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM ReviewPub Date : 2026-08-06DOI: 10.1137/26m1838982
Anita T. Layton
{"title":"Book Review:; Unlocking LaTeX Graphics: A Concise Guide to TikZ/PGF and PGFPLOTS","authors":"Anita T. Layton","doi":"10.1137/26m1838982","DOIUrl":"https://doi.org/10.1137/26m1838982","url":null,"abstract":"SIAM Review, Volume 68, Issue 3, Page 692-694, September 2026. <br/> Unlocking LaTeX Graphics addresses a well-known and persistent challenge in scientific and mathematical writing: producing high-quality, fully integrated figures directly within LaTeX using TikZ/PGF and PGFPLOTS. These packages are extraordinarily powerful, but their official documentation—spanning well over 1,800 pages—is daunting, even for experienced users. The stated goal of the book is therefore sharply defined and pragmatic: to distill the essential “10% of commands used 90% of the time” into a compact, coherent, and readable reference that enables users to become productive quickly, while also serving as a long-term desk reference.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"68 5 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2026-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148687911","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM ReviewPub Date : 2026-08-06DOI: 10.1137/25m1809173
Iguer L. D. Santos
{"title":"Book Review:; Computational Methods in Optimal Control: Theory and Practice","authors":"Iguer L. D. Santos","doi":"10.1137/25m1809173","DOIUrl":"https://doi.org/10.1137/25m1809173","url":null,"abstract":"SIAM Review, Volume 68, Issue 3, Page 690-691, September 2026. <br/> This is a book based on a series of 10 lectures (entitled Computational Methods in Optimal Control) given by the author at an NSF-CBMS conference. The book addresses the optimal control problem [math] for almost every [math], where [math] is given, [math], [math], [math] is bounded and measurable, and [math] is a closed set. Thus, the problem is time invariant on a fixed time interval [math] with control constraints and with vector-valued state and control.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"29 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2026-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148675292","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM ReviewPub Date : 2026-08-06DOI: 10.1137/24m1658875
Junaid Akhter, Paul David Fährmann, Konstantin Sonntag, Sebastian Peitz, Daniel Schwietert
{"title":"Common Pitfalls to Avoid While Using Multiobjective Optimization in Machine Learning","authors":"Junaid Akhter, Paul David Fährmann, Konstantin Sonntag, Sebastian Peitz, Daniel Schwietert","doi":"10.1137/24m1658875","DOIUrl":"https://doi.org/10.1137/24m1658875","url":null,"abstract":"SIAM Review, Volume 68, Issue 3, Page 656-683, September 2026. <br/> Abstract.Recently, there has been increasing interest in the application of multiobjective optimization (MOO) in machine learning (ML). This interest is driven by the numerous real-life situations in which multiple objectives must be optimized simultaneously. A key aspect of MOO is the existence of a Pareto set, rather than a single optimal solution, which represents the optimal trade-offs between different objectives. Despite its potential, there is a noticeable lack of satisfactory literature serving as an entry-level guide for ML practitioners aiming to apply MOO effectively. In this paper, our goal is to provide such a resource and highlight pitfalls to avoid. We begin by establishing the groundwork for MOO, focusing on well-known approaches such as the weighted sum (WS) method alongside more advanced techniques like the multiobjective gradient descent algorithm (MGDA). We critically review existing studies across various ML fields in which MOO has been applied and identify challenges that can lead to incorrect interpretations. One of these fields is physics informed neural networks (PINNs), which we use as a guiding example to carefully construct experiments illustrating these pitfalls. By comparing WS and MGDA with one of the most common evolutionary algorithms, NSGA-II, we demonstrate that difficulties can arise regardless of the specific MOO method used. We emphasize the importance of understanding the specific problem, the objective space, and the selected MOO method, while also noting that the neglect of factors such as convergence criteria can result in misleading experiments.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"54 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2026-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148687779","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM ReviewPub Date : 2026-08-06DOI: 10.1137/25m1747440
J. A. C. Weideman, Nicholas Hale
{"title":"Advection, Diffusion, Dispersion and All Things in Between","authors":"J. A. C. Weideman, Nicholas Hale","doi":"10.1137/25m1747440","DOIUrl":"https://doi.org/10.1137/25m1747440","url":null,"abstract":"SIAM Review, Volume 68, Issue 3, Page 637-655, September 2026. <br/> Abstract.As an example in their complex variables textbook, Mark Ablowitz and Athanassios Fokas apply the method of steepest descent to derive an asymptotic estimate for the solution of the linear Korteweg–de Vries (KdV) equation for large times. Here the analysis is extended to an equation with a fractional dispersion relation that contains this KdV equation (which models dispersion), the heat equation (diffusion), and the one-way wave equation (advection) as special cases.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"19 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2026-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148687780","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
SIAM ReviewPub Date : 2026-08-06DOI: 10.1137/25m1823210
Eduardo Casas
{"title":"Featured Review:; Numerical Analysis for Elliptic Optimal Control Problems","authors":"Eduardo Casas","doi":"10.1137/25m1823210","DOIUrl":"https://doi.org/10.1137/25m1823210","url":null,"abstract":"SIAM Review, Volume 68, Issue 3, Page 687-689, September 2026. <br/> Two fundamental issues arise in the study of an optimal control problem. First, we aim to establish the well-posedness of the problem, meaning that it has at least one solution. While the uniqueness of the solution is interesting, it is not crucial. The analysis of the state equation—a partial differential equation in the book at hand—is the starting point in proving the existence of an optimal control. Essential to this analysis is proving the existence and uniqueness of a solution to the state equation for every control and establishing the continuity of the control-to-state relation. The differentiability of the control-to-state relation is also essential for subsequent analyses. Some multistate control problems exist where the existence or uniqueness of a state corresponding to certain controls may fail, and this requires a different approach. However, such problems are not addressed in the present book.","PeriodicalId":49525,"journal":{"name":"SIAM Review","volume":"57 1","pages":""},"PeriodicalIF":10.2,"publicationDate":"2026-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"148688002","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}