Applied and Computational Harmonic Analysis最新文献

Controlled learning of pointwise nonlinearities in neural-network-like architectures

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-03-25 DOI: 10.1016/j.acha.2025.101764

Michael Unser, Alexis Goujon, Stanislas Ducotterd

{"title":"Controlled learning of pointwise nonlinearities in neural-network-like architectures","authors":"Michael Unser, Alexis Goujon, Stanislas Ducotterd","doi":"10.1016/j.acha.2025.101764","DOIUrl":"10.1016/j.acha.2025.101764","url":null,"abstract":"<div><div>We present a general variational framework for the training of freeform nonlinearities in layered computational architectures subject to some slope constraints. The regularization that we add to the traditional training loss penalizes the second-order total variation of each trainable activation. The slope constraints allow us to impose properties such as 1-Lipschitz stability, firm non-expansiveness, and monotonicity/invertibility. These properties are crucial to ensure the proper functioning of certain classes of signal-processing algorithms (e.g., plug-and-play schemes, unrolled proximal gradient, invertible flows). We prove that the global optimum of the stated constrained-optimization problem is achieved with nonlinearities that are adaptive nonuniform linear splines. We then show how to solve the resulting function-optimization problem numerically by representing the nonlinearities in a suitable (nonuniform) B-spline basis. Finally, we illustrate the use of our framework with the data-driven design of (weakly) convex regularizers for the denoising of images and the resolution of inverse problems.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"77 ","pages":"Article 101764"},"PeriodicalIF":2.6,"publicationDate":"2025-03-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143738438","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Mathematical algorithm design for deep learning under societal and judicial constraints: The algorithmic transparency requirement

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-03-24 DOI: 10.1016/j.acha.2025.101763

Holger Boche , Adalbert Fono , Gitta Kutyniok

{"title":"Mathematical algorithm design for deep learning under societal and judicial constraints: The algorithmic transparency requirement","authors":"Holger Boche , Adalbert Fono , Gitta Kutyniok","doi":"10.1016/j.acha.2025.101763","DOIUrl":"10.1016/j.acha.2025.101763","url":null,"abstract":"<div><div>Deep learning still has drawbacks regarding trustworthiness, which describes a comprehensible, fair, safe, and reliable method. To mitigate the potential risk of AI, clear obligations associated with trustworthiness have been proposed via regulatory guidelines, e.g., in the European AI Act. Therefore, a central question is to what extent trustworthy deep learning can be realized. Establishing the described properties constituting trustworthiness requires that the factors influencing an algorithmic computation can be retraced, i.e., the algorithmic implementation is transparent. Motivated by the observation that the current evolution of deep learning models necessitates a change in computing technology, we derive a mathematical framework that enables us to analyze whether a transparent implementation in a computing model is feasible. The core idea is to formalize and subsequently relate the properties of a transparent algorithmic implementation to the mathematical model of the computing platform, thereby establishing verifiable criteria.</div><div>We exemplarily apply our trustworthiness framework to analyze deep learning approaches for inverse problems in digital and analog computing models represented by Turing and Blum-Shub-Smale machines, respectively. Based on previous results, we find that Blum-Shub-Smale machines have the potential to establish trustworthy solvers for inverse problems under fairly general conditions, whereas Turing machines cannot guarantee trustworthiness to the same degree.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"77 ","pages":"Article 101763"},"PeriodicalIF":2.6,"publicationDate":"2025-03-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143696911","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An efficient spatial discretization of spans of multivariate Chebyshev polynomials

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-03-20 DOI: 10.1016/j.acha.2025.101761

Lutz Kämmerer

{"title":"An efficient spatial discretization of spans of multivariate Chebyshev polynomials","authors":"Lutz Kämmerer","doi":"10.1016/j.acha.2025.101761","DOIUrl":"10.1016/j.acha.2025.101761","url":null,"abstract":"<div><div>For an arbitrary given span of high dimensional multivariate Chebyshev polynomials, an approach to construct spatial discretizations is presented, i.e., the construction of a sampling set that allows for the unique reconstruction of each polynomial of this span.</div><div>The approach presented here combines three different types of efficiency. First, the construction of a spatial discretization should be computationally efficient with respect to the dimension of the span of the Chebyshev polynomials. Second, the constructed discretization should be sample efficient, i.e., the number of sampling nodes within the constructed discretization should be reasonably low. Third, there should be an efficient algorithm for the unique reconstruction of a polynomial from given sampling values at the sampling nodes of the discretization.</div><div>The first two mentioned types of efficiency are also present in constructions based on random sampling nodes, but the lack of structure here causes the inefficiency of the reconstruction method. Our approach uses a combination of cosine transformed rank-1 lattices whose structure allows for applications of univariate fast Fourier transforms for the reconstruction algorithm and is thus a priori efficiently realizable.</div><div>Besides the theoretical estimates of numbers of sampling nodes and failure probabilities due to a random draw of the used lattices, we present several improvements of the basic design approach that significantly increases its practical applicability. Numerical tests, which discretize spans of multivariate Chebyshev polynomials depending on up to more than 50 spatial variables, corroborate the theoretical results and the significance of the improvements.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"77 ","pages":"Article 101761"},"PeriodicalIF":2.6,"publicationDate":"2025-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143697625","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

An inverse problem for Dirac systems on p-star-shaped graphs

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-03-20 DOI: 10.1016/j.acha.2025.101760

Yu Ping Wang , Yan-Hsiou Cheng

引用次数: 0

Error estimate of the u-series method for molecular dynamics simulations

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-03-14 DOI: 10.1016/j.acha.2025.101759

Jiuyang Liang , Zhenli Xu , Qi Zhou

引用次数: 0

The Large Deviation Principle for W-random spectral measures

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-02-21 DOI: 10.1016/j.acha.2025.101756

Mahya Ghandehari , Georgi S. Medvedev

引用次数: 0

Efficient identification of wide shallow neural networks with biases

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-02-17 DOI: 10.1016/j.acha.2025.101749

Massimo Fornasier , Timo Klock , Marco Mondelli , Michael Rauchensteiner

{"title":"Efficient identification of wide shallow neural networks with biases","authors":"Massimo Fornasier , Timo Klock , Marco Mondelli , Michael Rauchensteiner","doi":"10.1016/j.acha.2025.101749","DOIUrl":"10.1016/j.acha.2025.101749","url":null,"abstract":"<div><div>The identification of the parameters of a neural network from finite samples of input-output pairs is often referred to as the <em>teacher-student model</em>, and this model has represented a popular framework for understanding training and generalization. Even if the problem is NP-complete in the worst case, a rapidly growing literature – after adding suitable distributional assumptions – has established finite sample identification of two-layer networks with a number of neurons <span><math><mi>m</mi><mo>=</mo><mi>O</mi><mo>(</mo><mi>D</mi><mo>)</mo></math></span>, <em>D</em> being the input dimension. For the range <span><math><mi>D</mi><mo><</mo><mi>m</mi><mo><</mo><msup><mrow><mi>D</mi></mrow><mrow><mn>2</mn></mrow></msup></math></span> the problem becomes harder, and truly little is known for networks parametrized by biases as well. This paper fills the gap by providing efficient algorithms and rigorous theoretical guarantees of finite sample identification for such wider shallow networks with biases. Our approach is based on a two-step pipeline: first, we recover the direction of the weights, by exploiting second order information; next, we identify the signs by suitable algebraic evaluations, and we recover the biases by empirical risk minimization via gradient descent. Numerical results demonstrate the effectiveness of our approach.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"77 ","pages":"Article 101749"},"PeriodicalIF":2.6,"publicationDate":"2025-02-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143429952","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Kadec-type theorems for sampled group orbits 抽样群轨道的kadec型定理

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-01-10 DOI: 10.1016/j.acha.2025.101748

Ilya Krishtal, Brendan Miller

引用次数: 0

On the non-frame property of Gabor systems with Hermite generators and the frame set conjecture 带有Hermite发生器的Gabor系统的非框架性质及框架集猜想

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-01-10 DOI: 10.1016/j.acha.2025.101747

Andreas Horst , Jakob Lemvig , Allan Erlang Videbæk

引用次数: 0

How robust is randomized blind deconvolution via nuclear norm minimization against adversarial noise? 通过核范数最小化随机盲反卷积对对抗噪声的鲁棒性如何？

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2024-12-30 DOI: 10.1016/j.acha.2024.101746

Julia Kostin , Felix Krahmer , Dominik Stöger

{"title":"How robust is randomized blind deconvolution via nuclear norm minimization against adversarial noise?","authors":"Julia Kostin , Felix Krahmer , Dominik Stöger","doi":"10.1016/j.acha.2024.101746","DOIUrl":"10.1016/j.acha.2024.101746","url":null,"abstract":"<div><div>In this paper, we study the problem of recovering two unknown signals from their convolution, which is commonly referred to as blind deconvolution. Reformulation of blind deconvolution as a low-rank recovery problem has led to multiple theoretical recovery guarantees in the past decade due to the success of the nuclear norm minimization heuristic. In particular, in the absence of noise, exact recovery has been established for sufficiently incoherent signals contained in lower-dimensional subspaces. However, if the convolution is corrupted by additive bounded noise, the stability of the recovery problem remains much less understood. In particular, existing reconstruction bounds involve large dimension factors and therefore fail to explain the empirical evidence for dimension-independent robustness of nuclear norm minimization. Recently, theoretical evidence has emerged for ill-posed behaviour of low-rank matrix recovery for sufficiently small noise levels. In this work, we develop improved recovery guarantees for blind deconvolution with adversarial noise which exhibit square-root scaling in the noise level. Hence, our results are consistent with existing counterexamples which speak against linear scaling in the noise level as demonstrated for related low-rank matrix recovery problems.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"76 ","pages":"Article 101746"},"PeriodicalIF":2.6,"publicationDate":"2024-12-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142968152","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0