Applied and Computational Harmonic Analysis最新文献

Minibatch and local SGD: Algorithmic stability and linear speedup in generalization 小批量和局部SGD：算法的稳定性和泛化的线性加速

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-07-16 DOI: 10.1016/j.acha.2025.101795

Yunwen Lei , Tao Sun , Mingrui Liu

引用次数: 0

Multi-dimensional unlimited sampling and robust reconstruction 多维无限采样和鲁棒重建

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-07-16 DOI: 10.1016/j.acha.2025.101796

Dorian Florescu, Ayush Bhandari

{"title":"Multi-dimensional unlimited sampling and robust reconstruction","authors":"Dorian Florescu, Ayush Bhandari","doi":"10.1016/j.acha.2025.101796","DOIUrl":"10.1016/j.acha.2025.101796","url":null,"abstract":"<div><div>In this paper we introduce a new sampling and reconstruction approach for multi-dimensional analog signals. Building on top of the Unlimited Sensing Framework (USF), we present a new folded sampling operator called the multi-dimensional modulo-hysteresis that is also backwards compatible with the existing one-dimensional modulo operator. Unlike previous approaches, the proposed model is specifically tailored to multi-dimensional signals. In particular, the model uses certain redundancy in dimensions 2 and above, which is exploited for input recovery with robustness. We prove that the new operator is well-defined and its outputs have a bounded dynamic range. For the noiseless case, we derive a theoretically guaranteed input reconstruction approach. When the input is corrupted by Gaussian noise, we exploit redundancy in higher dimensions to provide a bound on the error probability and show this drops to 0 for high enough sampling rates leading to new theoretical guarantees for the noisy case. Our numerical examples corroborate the theoretical results and show that the proposed approach can handle a significantly larger amount of noise compared to USF.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"79 ","pages":"Article 101796"},"PeriodicalIF":2.6,"publicationDate":"2025-07-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144665025","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networks 基于深度ReLU神经网络的Sobolev和Besov函数的最优逼近

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-07-16 DOI: 10.1016/j.acha.2025.101797

Yunfei Yang

{"title":"On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networks","authors":"Yunfei Yang","doi":"10.1016/j.acha.2025.101797","DOIUrl":"10.1016/j.acha.2025.101797","url":null,"abstract":"<div><div>This paper studies the problem of how efficiently functions in the Sobolev spaces <span><math><msup><mrow><mi>W</mi></mrow><mrow><mi>s</mi><mo>,</mo><mi>q</mi></mrow></msup><mo>(</mo><msup><mrow><mo>[</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo>]</mo></mrow><mrow><mi>d</mi></mrow></msup><mo>)</mo></math></span> and Besov spaces <span><math><msubsup><mrow><mi>B</mi></mrow><mrow><mi>q</mi><mo>,</mo><mi>r</mi></mrow><mrow><mi>s</mi></mrow></msubsup><mo>(</mo><msup><mrow><mo>[</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo>]</mo></mrow><mrow><mi>d</mi></mrow></msup><mo>)</mo></math></span> can be approximated by deep ReLU neural networks with width <em>W</em> and depth <em>L</em>, when the error is measured in the <span><math><msup><mrow><mi>L</mi></mrow><mrow><mi>p</mi></mrow></msup><mo>(</mo><msup><mrow><mo>[</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo>]</mo></mrow><mrow><mi>d</mi></mrow></msup><mo>)</mo></math></span> norm. This problem has been studied by several recent works, which obtained the approximation rate <span><math><mi>O</mi><mo>(</mo><msup><mrow><mo>(</mo><mi>W</mi><mi>L</mi><mo>)</mo></mrow><mrow><mo>−</mo><mn>2</mn><mi>s</mi><mo>/</mo><mi>d</mi></mrow></msup><mo>)</mo></math></span> up to logarithmic factors when <span><math><mi>p</mi><mo>=</mo><mi>q</mi><mo>=</mo><mo>∞</mo></math></span>, and the rate <span><math><mi>O</mi><mo>(</mo><msup><mrow><mi>L</mi></mrow><mrow><mo>−</mo><mn>2</mn><mi>s</mi><mo>/</mo><mi>d</mi></mrow></msup><mo>)</mo></math></span> for networks with fixed width when the Sobolev embedding condition <span><math><mn>1</mn><mo>/</mo><mi>q</mi><mo>−</mo><mn>1</mn><mo>/</mo><mi>p</mi><mo><</mo><mi>s</mi><mo>/</mo><mi>d</mi></math></span> holds. We generalize these results by showing that the rate <span><math><mi>O</mi><mo>(</mo><msup><mrow><mo>(</mo><mi>W</mi><mi>L</mi><mo>)</mo></mrow><mrow><mo>−</mo><mn>2</mn><mi>s</mi><mo>/</mo><mi>d</mi></mrow></msup><mo>)</mo></math></span> indeed holds under the Sobolev embedding condition. It is known that this rate is optimal up to logarithmic factors. The key tool in our proof is a novel encoding of sparse vectors by using deep ReLU neural networks with varied width and depth, which may be of independent interest.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"79 ","pages":"Article 101797"},"PeriodicalIF":2.6,"publicationDate":"2025-07-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144653252","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Nonharmonic multivariate Fourier transforms and matrices: Condition numbers and hyperplane geometry 非调和多元傅立叶变换与矩阵：条件数与超平面几何

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-07-09 DOI: 10.1016/j.acha.2025.101791

Weilin Li

{"title":"Nonharmonic multivariate Fourier transforms and matrices: Condition numbers and hyperplane geometry","authors":"Weilin Li","doi":"10.1016/j.acha.2025.101791","DOIUrl":"10.1016/j.acha.2025.101791","url":null,"abstract":"<div><div>Consider an operator that takes the Fourier transform of a discrete measure supported in <span><math><mi>X</mi><mo>⊆</mo><msup><mrow><mo>[</mo><mo>−</mo><mfrac><mrow><mn>1</mn></mrow><mrow><mn>2</mn></mrow></mfrac><mo>,</mo><mfrac><mrow><mn>1</mn></mrow><mrow><mn>2</mn></mrow></mfrac><mo>)</mo></mrow><mrow><mi>d</mi></mrow></msup></math></span> and restricts it to a compact <span><math><mi>Ω</mi><mo>⊆</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>d</mi></mrow></msup></math></span>. We provide lower bounds for its smallest singular value when Ω is either a closed ball of radius <em>m</em> or closed cube of side length 2<em>m</em>, and under different types of geometric assumptions on <span><math><mi>X</mi></math></span>. We first show that if distances between points in <span><math><mi>X</mi></math></span> are lower bounded by a <em>δ</em> that is allowed to be arbitrarily small, then the smallest singular value is at least <span><math><mi>C</mi><msup><mrow><mi>m</mi></mrow><mrow><mi>d</mi><mo>/</mo><mn>2</mn></mrow></msup><msup><mrow><mo>(</mo><mi>m</mi><mi>δ</mi><mo>)</mo></mrow><mrow><mi>λ</mi><mo>−</mo><mn>1</mn></mrow></msup></math></span>, where <em>λ</em> is the maximum number of elements in <span><math><mi>X</mi></math></span> contained within any ball or cube of an explicitly given radius. This estimate communicates a localization effect of the Fourier transform. While it is sharp, the smallest singular value behaves better than expected for many <span><math><mi>X</mi></math></span>, including when we dilate a generic set by parameter <em>δ</em>. We next show that if there is a <em>η</em> such that, for each <span><math><mi>x</mi><mo>∈</mo><mi>X</mi></math></span>, the set <span><math><mi>X</mi><mo>∖</mo><mo>{</mo><mi>x</mi><mo>}</mo></math></span> locally consists of at most <em>r</em> hyperplanes whose distances to <em>x</em> are at least <em>η</em>, then the smallest singular value is at least <span><math><mi>C</mi><msup><mrow><mi>m</mi></mrow><mrow><mi>d</mi><mo>/</mo><mn>2</mn></mrow></msup><msup><mrow><mo>(</mo><mi>m</mi><mi>η</mi><mo>)</mo></mrow><mrow><mi>r</mi></mrow></msup></math></span>. For dilations of a generic set by <em>δ</em>, the lower bound becomes <span><math><mi>C</mi><msup><mrow><mi>m</mi></mrow><mrow><mi>d</mi><mo>/</mo><mn>2</mn></mrow></msup><msup><mrow><mo>(</mo><mi>m</mi><mi>δ</mi><mo>)</mo></mrow><mrow><mo>⌈</mo><mo>(</mo><mi>λ</mi><mo>−</mo><mn>1</mn><mo>)</mo><mo>/</mo><mi>d</mi><mo>⌉</mo></mrow></msup></math></span>. The appearance of a <span><math><mn>1</mn><mo>/</mo><mi>d</mi></math></span> factor in the exponent indicates that compared to worst case scenarios, the condition number of nonharmonic Fourier transforms is better than expected for typical sets and improve with higher dimensionality.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"79 ","pages":"Article 101791"},"PeriodicalIF":2.6,"publicationDate":"2025-07-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144595712","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

On exact systems {tα⋅e2πint}n∈Z∖A in L2(0,1) which are weighted lower semi frames but not Schauder bases, and their generalizations 关于L2（0,1）中为加权下半坐标系但非Schauder基的精确系统{tα⋅e2πint}n∈Z∈A及其推广

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-07-09 DOI: 10.1016/j.acha.2025.101794

Elias Zikkos

引用次数: 0

On the limits of neural network explainability via descrambling 解扰论神经网络可解释性的局限性

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-07-07 DOI: 10.1016/j.acha.2025.101793

Shashank Sule , Richard G. Spencer , Wojciech Czaja

{"title":"On the limits of neural network explainability via descrambling","authors":"Shashank Sule , Richard G. Spencer , Wojciech Czaja","doi":"10.1016/j.acha.2025.101793","DOIUrl":"10.1016/j.acha.2025.101793","url":null,"abstract":"<div><div>We characterize the exact solutions to <em>neural network descrambling</em>–a mathematical model for explaining the fully connected layers of trained neural networks (NNs). By reformulating the problem to the minimization of the Brockett function arising in graph matching and complexity theory we show that the principal components of the hidden layer preactivations can be characterized as the optimal “explainers” or <em>descramblers</em> for the layer weights, leading to <em>descrambled</em> weight matrices. We show that in typical deep learning contexts these descramblers take diverse and interesting forms including (1) matching largest principal components with the lowest frequency modes of the Fourier basis for isotropic hidden data, (2) discovering the semantic development in two-layer linear NNs for signal recovery problems, and (3) explaining CNNs by optimally permuting the neurons. Our numerical experiments indicate that the eigendecompositions of the hidden layer data–now understood as the descramblers–can also reveal the layer's underlying transformation. These results illustrate that the SVD is more directly related to the explainability of NNs than previously thought and offers a promising avenue for discovering interpretable motifs for the hidden action of NNs, especially in contexts of operator learning or physics-informed NNs, where the input/output data has limited human readability.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"79 ","pages":"Article 101793"},"PeriodicalIF":2.6,"publicationDate":"2025-07-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144572145","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Gaussian process regression with log-linear scaling for common non-stationary kernels 常见非平稳核的对数线性标度高斯过程回归

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-07-05 DOI: 10.1016/j.acha.2025.101792

P. Michael Kielstra , Michael Lindsey

{"title":"Gaussian process regression with log-linear scaling for common non-stationary kernels","authors":"P. Michael Kielstra , Michael Lindsey","doi":"10.1016/j.acha.2025.101792","DOIUrl":"10.1016/j.acha.2025.101792","url":null,"abstract":"<div><div>We introduce a fast algorithm for Gaussian process regression in low dimensions, applicable to a widely-used family of non-stationary kernels. The non-stationarity of these kernels is induced by arbitrary spatially-varying vertical and horizontal scales. In particular, any stationary kernel can be accommodated as a special case, and we focus especially on the generalization of the standard Matérn kernel. Our subroutine for kernel matrix-vector multiplications scales almost optimally as <span><math><mi>O</mi><mo>(</mo><mi>N</mi><mi>log</mi><mo>⁡</mo><mi>N</mi><mo>)</mo></math></span>, where <em>N</em> is the number of regression points. Like the recently developed equispaced Fourier Gaussian process (EFGP) methodology, which is applicable only to stationary kernels, our approach exploits non-uniform fast Fourier transforms (NUFFTs). We offer a complete analysis controlling the approximation error of our method, and we validate the method's practical performance with numerical experiments. In particular we demonstrate improved scalability compared to state-of-the-art rank-structured approaches in spatial dimension <span><math><mi>d</mi><mo>></mo><mn>1</mn></math></span>.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"79 ","pages":"Article 101792"},"PeriodicalIF":2.6,"publicationDate":"2025-07-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144596067","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Energy propagation in scattering convolution networks can be arbitrarily slow 散射卷积网络中的能量传播可以是任意慢的

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-06-20 DOI: 10.1016/j.acha.2025.101790

Hartmut Führ, Max Getter

{"title":"Energy propagation in scattering convolution networks can be arbitrarily slow","authors":"Hartmut Führ, Max Getter","doi":"10.1016/j.acha.2025.101790","DOIUrl":"10.1016/j.acha.2025.101790","url":null,"abstract":"<div><div>We analyze energy decay for deep convolutional neural networks employed as feature extractors, including Mallat's wavelet scattering transform. For time-frequency scattering transforms based on Gabor filters, previous work has established that energy decay is exponential for arbitrary square-integrable input signals. In contrast, our main results allow proving that this is false for wavelet scattering in any dimension. Specifically, we show that the energy decay of wavelet and wavelet-like scattering transforms acting on generic square-integrable signals can be arbitrarily slow. Importantly, this slow decay behavior holds for dense subsets of <span><math><msup><mrow><mi>L</mi></mrow><mrow><mn>2</mn></mrow></msup><mo>(</mo><msup><mrow><mi>R</mi></mrow><mrow><mi>d</mi></mrow></msup><mo>)</mo></math></span>, indicating that rapid energy decay is generally an unstable property of signals. We complement these findings with positive results that allow us to infer fast (up to exponential) energy decay for generalized Sobolev spaces tailored to the frequency localization of the underlying filter bank. Both negative and positive results highlight that energy decay in scattering networks critically depends on the interplay between the respective frequency localizations of both the signal and the filters used.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"79 ","pages":"Article 101790"},"PeriodicalIF":2.6,"publicationDate":"2025-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144337762","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

ANOVA-boosting for random Fourier features 随机傅里叶特征的anova增强

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-06-18 DOI: 10.1016/j.acha.2025.101789

Daniel Potts, Laura Weidensager

引用次数: 0

New results on sparse representations in unions of orthonormal bases 标准正交基并中的稀疏表示的新结果

IF 2.6 2区数学

Applied and Computational Harmonic Analysis Pub Date : 2025-06-11 DOI: 10.1016/j.acha.2025.101786

Tao Zhang , Gennian Ge

{"title":"New results on sparse representations in unions of orthonormal bases","authors":"Tao Zhang , Gennian Ge","doi":"10.1016/j.acha.2025.101786","DOIUrl":"10.1016/j.acha.2025.101786","url":null,"abstract":"<div><div>The problem of sparse representation has significant applications in signal processing. The spark of a dictionary plays a crucial role in the study of sparse representation. Donoho and Elad initially explored the spark, and they provided a general lower bound. When the dictionary is a union of several orthonormal bases, Gribonval and Nielsen presented an improved lower bound for spark. In this paper, we introduce a new construction of dictionary, achieving the spark bound given by Gribonval and Nielsen. More precisely, let <em>q</em> be a power of 2, we show that for any positive integer <em>t</em>, there exists a dictionary in <span><math><msup><mrow><mi>R</mi></mrow><mrow><msup><mrow><mi>q</mi></mrow><mrow><mn>2</mn><mi>t</mi></mrow></msup></mrow></msup></math></span>, which is a union of <span><math><mi>q</mi><mo>+</mo><mn>1</mn></math></span> orthonormal bases, such that the spark of the dictionary attains Gribonval-Nielsen's bound. Our result extends previously best known result from <span><math><mi>t</mi><mo>=</mo><mn>1</mn><mo>,</mo><mn>2</mn></math></span> to arbitrarily positive integer <em>t</em>, and our construction is technically different from previous ones. Their method is more combinatorial, while ours is algebraic, which is more general.</div></div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"79 ","pages":"Article 101786"},"PeriodicalIF":2.6,"publicationDate":"2025-06-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144262223","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0