A multivariate Riesz basis of ReLU neural networks

IF 2.6 2区数学 Q1 MATHEMATICS, APPLIED

Applied and Computational Harmonic Analysis Pub Date : 2023-10-20 DOI:10.1016/j.acha.2023.101605

Cornelia Schneider , Jan Vybíral

{"title":"A multivariate Riesz basis of ReLU neural networks","authors":"Cornelia Schneider , Jan Vybíral","doi":"10.1016/j.acha.2023.101605","DOIUrl":null,"url":null,"abstract":"<div>We consider the trigonometric-like system of piecewise linear functions introduced recently by Daubechies, DeVore, Foucart, Hanin, and Petrova. We provide an alternative proof that this system forms a Riesz basis of <math><msub><mrow><mi>L</mi></mrow><mrow><mn>2</mn></mrow></msub><mo>(</mo><mo>[</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo>]</mo><mo>)</mo></math> based on the Gershgorin theorem. We also generalize this system to higher dimensions <math><mi>d</mi><mo>></mo><mn>1</mn></math> by a construction, which avoids using (tensor) products. As a consequence, the functions from the new Riesz basis of <math><msub><mrow><mi>L</mi></mrow><mrow><mn>2</mn></mrow></msub><mo>(</mo><msup><mrow><mo>[</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo>]</mo></mrow><mrow><mi>d</mi></mrow></msup><mo>)</mo></math> can be easily represented by neural networks. Moreover, the Riesz constants of this system are independent of d, making it an attractive building block regarding future multivariate analysis of neural networks.</div>","PeriodicalId":55504,"journal":{"name":"Applied and Computational Harmonic Analysis","volume":"68 ","pages":"Article 101605"},"PeriodicalIF":2.6000,"publicationDate":"2023-10-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Applied and Computational Harmonic Analysis","FirstCategoryId":"100","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1063520323000921","RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MATHEMATICS, APPLIED","Score":null,"Total":0}

引用次数: 0

Abstract

We consider the trigonometric-like system of piecewise linear functions introduced recently by Daubechies, DeVore, Foucart, Hanin, and Petrova. We provide an alternative proof that this system forms a Riesz basis of $L_{2} ([0, 1])$ based on the Gershgorin theorem. We also generalize this system to higher dimensions $d > 1$ by a construction, which avoids using (tensor) products. As a consequence, the functions from the new Riesz basis of $L_{2} ({[0, 1]}^{d})$ can be easily represented by neural networks. Moreover, the Riesz constants of this system are independent of d, making it an attractive building block regarding future multivariate analysis of neural networks.

查看原文本刊更多论文

ReLU神经网络的多元Riesz基

我们考虑Daubechies、DeVore、Foucart、Hanin和Petrova最近提出的分段线性函数的类三角系统。基于Gershgorin定理，我们提供了另一个证明，证明该系统形成了L2（[0,1]）的Riesz基。我们还将该系统推广到更高维度d>；1通过避免使用（张量）乘积的构造。因此，L2（[0,1]d）的新Riesz基的函数可以很容易地用神经网络表示。此外，该系统的Riesz常数与d无关，这使其成为未来神经网络多变量分析的一个有吸引力的构建块。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Applied and Computational Harmonic Analysis 物理-物理：数学物理

CiteScore

5.40

自引率

4.00%

发文量

审稿时长

22.9 weeks

期刊介绍： Applied and Computational Harmonic Analysis (ACHA) is an interdisciplinary journal that publishes high-quality papers in all areas of mathematical sciences related to the applied and computational aspects of harmonic analysis, with special emphasis on innovative theoretical development, methods, and algorithms, for information processing, manipulation, understanding, and so forth. The objectives of the journal are to chronicle the important publications in the rapidly growing field of data representation and analysis, to stimulate research in relevant interdisciplinary areas, and to provide a common link among mathematical, physical, and life scientists, as well as engineers.