SIAM journal on mathematics of data science最新文献_第9页

Generalization error of minimum weighted norm and kernel interpolation 最小加权范数与核插值的泛化误差

SIAM journal on mathematics of data science Pub Date : 2020-08-07 DOI: 10.1137/20M1359912

Weilin Li

引用次数: 7

Normal-bundle Bootstrap 法丛引导

SIAM journal on mathematics of data science Pub Date : 2020-07-27 DOI: 10.1137/20M1356002

Ruda Zhang, R. Ghanem

引用次数: 3

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection 像(Var)Pro一样训练:具有可变投影的神经网络的有效训练

SIAM journal on mathematics of data science Pub Date : 2020-07-26 DOI: 10.1137/20m1359511

Elizabeth Newman, Lars Ruthotto, Joseph L. Hart, B. V. B. Waanders

{"title":"Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection","authors":"Elizabeth Newman, Lars Ruthotto, Joseph L. Hart, B. V. B. Waanders","doi":"10.1137/20m1359511","DOIUrl":"https://doi.org/10.1137/20m1359511","url":null,"abstract":"Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations (PDE) and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, non-convexity, data-sparsity, and non-trivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss-Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to non-quadratic objective functions, most notably, cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In numerical experiments from classification and surrogate modeling, GNvpro not only solves the optimization problem more efficiently but also yields DNNs that generalize better than commonly-used optimization schemes.","PeriodicalId":74797,"journal":{"name":"SIAM journal on mathematics of data science","volume":"10 2 1","pages":"1041-1066"},"PeriodicalIF":0.0,"publicationDate":"2020-07-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"81647654","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 14

EnResNet: ResNets Ensemble via the Feynman-Kac Formalism for Adversarial Defense and Beyond EnResNet:基于费曼-卡茨形式主义的对抗防御及其后续的ResNets集成

SIAM journal on mathematics of data science Pub Date : 2020-07-13 DOI: 10.1137/19m1265302

Bao Wang, Binjie Yuan, Zuoqiang Shi, S. Osher

引用次数: 8

A Performance Guarantee for Spectral Clustering 谱聚类的性能保证

SIAM journal on mathematics of data science Pub Date : 2020-07-10 DOI: 10.1137/20M1352193

M. Boedihardjo, Shaofeng Deng, T. Strohmer

引用次数: 5

Semi-supervised Learning for Aggregated Multilayer Graphs Using Diffuse Interface Methods and Fast Matrix-Vector Products 基于扩散接口方法和快速矩阵向量积的聚合多层图半监督学习

SIAM journal on mathematics of data science Pub Date : 2020-07-10 DOI: 10.1137/20M1352028

Kai Bergermann, M. Stoll, Toni Volkmer

{"title":"Semi-supervised Learning for Aggregated Multilayer Graphs Using Diffuse Interface Methods and Fast Matrix-Vector Products","authors":"Kai Bergermann, M. Stoll, Toni Volkmer","doi":"10.1137/20M1352028","DOIUrl":"https://doi.org/10.1137/20M1352028","url":null,"abstract":"We generalize a graph-based multiclass semi-supervised classification technique based on diffuse interface methods to multilayer graphs. Besides the treatment of various applications with an inherent multilayer structure, we present a very flexible approach that interprets high-dimensional data in a low-dimensional multilayer graph representation. Highly efficient numerical methods involving the spectral decomposition of the corresponding differential graph operators as well as fast matrix-vector products based on the nonequispaced fast Fourier transform (NFFT) enable the rapid treatment of large and high-dimensional data sets. We perform various numerical tests putting a special focus on image segmentation. In particular, we test the performance of our method on data sets with up to 10 million nodes per layer as well as up to 104 dimensions resulting in graphs with up to 52 layers. While all presented numerical experiments can be run on an average laptop computer, the linear dependence per iteration step of the runtime on the network size in all stages of our algorithm makes it scalable to even larger and higher-dimensional problems.","PeriodicalId":74797,"journal":{"name":"SIAM journal on mathematics of data science","volume":"80 1","pages":"758-785"},"PeriodicalIF":0.0,"publicationDate":"2020-07-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73724464","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 8

Variational Representations and Neural Network Estimation of Rényi Divergences rassanyi散度的变分表示与神经网络估计

SIAM journal on mathematics of data science Pub Date : 2020-07-07 DOI: 10.1137/20m1368926

Jeremiah Birrell, P. Dupuis, M. Katsoulakis, L. Rey-Bellet, Jie Wang

引用次数: 22

The Signature Kernel Is the Solution of a Goursat PDE 签名核是一个Goursat PDE的解

SIAM journal on mathematics of data science Pub Date : 2020-06-26 DOI: 10.1137/20M1366794

C. Salvi, Thomas Cass, J. Foster, Terry Lyons, Weixin Yang

{"title":"The Signature Kernel Is the Solution of a Goursat PDE","authors":"C. Salvi, Thomas Cass, J. Foster, Terry Lyons, Weixin Yang","doi":"10.1137/20M1366794","DOIUrl":"https://doi.org/10.1137/20M1366794","url":null,"abstract":"Recently, there has been an increased interest in the development of kernel methods for learning with sequential data. The signature kernel is a learning tool with potential to handle irregularly sampled, multivariate time series. In\"Kernels for sequentially ordered data\"the authors introduced a kernel trick for the truncated version of this kernel avoiding the exponential complexity that would have been involved in a direct computation. Here we show that for continuously differentiable paths, the signature kernel solves a hyperbolic PDE and recognize the connection with a well known class of differential equations known in the literature as Goursat problems. This Goursat PDE only depends on the increments of the input sequences, does not require the explicit computation of signatures and can be solved efficiently using state-of-the-arthyperbolic PDE numerical solvers, giving a kernel trick for the untruncated signature kernel, with the same raw complexity as the method from\"Kernels for sequentially ordered data\", but with the advantage that the PDE numerical scheme is well suited for GPU parallelization, which effectively reduces the complexity by a full order of magnitude in the length of the input sequences. In addition, we extend the previous analysis to the space of geometric rough paths and establish, using classical results from rough path theory, that the rough version of the signature kernel solves a rough integral equation analogous to the aforementioned Goursat PDE. Finally, we empirically demonstrate the effectiveness of our PDE kernel as a machine learning tool in various machine learning applications dealing with sequential data. We release the library sigkernel publicly available at https://github.com/crispitagorico/sigkernel.","PeriodicalId":74797,"journal":{"name":"SIAM journal on mathematics of data science","volume":"15 1","pages":"873-899"},"PeriodicalIF":0.0,"publicationDate":"2020-06-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"80665343","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 32

Memory-Efficient Structured Convex Optimization via Extreme Point Sampling 基于极值点抽样的高效内存结构凸优化

SIAM journal on mathematics of data science Pub Date : 2020-06-19 DOI: 10.1137/20m1358037

Nimita Shinde, Vishnu Narayanan, J. Saunderson

引用次数: 4

Two Steps at a Time---Taking GAN Training in Stride with Tseng's Method 一次两步——用曾氏方法进行GAN训练

SIAM journal on mathematics of data science Pub Date : 2020-06-16 DOI: 10.1137/21m1420939

A. Böhm, Michael Sedlmayer, E. R. Csetnek, R. Boț

引用次数: 13