Mathematical and Scientific Machine Learning最新文献

SpecNet2: Orthogonalization-free spectral embedding by neural networks SpecNet2:基于神经网络的无正交化谱嵌入

Mathematical and Scientific Machine Learning Pub Date : 2022-06-14 DOI: 10.48550/arXiv.2206.06644

Ziyu Chen, Yingzhou Li, Xiuyuan Cheng

{"title":"SpecNet2: Orthogonalization-free spectral embedding by neural networks","authors":"Ziyu Chen, Yingzhou Li, Xiuyuan Cheng","doi":"10.48550/arXiv.2206.06644","DOIUrl":"https://doi.org/10.48550/arXiv.2206.06644","url":null,"abstract":"Spectral methods which represent data points by eigenvectors of kernel matrices or graph Laplacian matrices have been a primary tool in unsupervised data analysis. In many application scenarios, parametrizing the spectral embedding by a neural network that can be trained over batches of data samples gives a promising way to achieve automatic out-of-sample extension as well as computational scalability. Such an approach was taken in the original paper of SpectralNet (Shaham et al. 2018), which we call SpecNet1. The current paper introduces a new neural network approach, named SpecNet2, to compute spectral embedding which optimizes an equivalent objective of the eigen-problem and removes the orthogonalization layer in SpecNet1. SpecNet2 also allows separating the sampling of rows and columns of the graph affinity matrix by tracking the neighbors of each data point through the gradient formula. Theoretically, we show that any local minimizer of the new orthogonalization-free objective reveals the leading eigenvectors. Furthermore, global convergence for this new orthogonalization-free objective using a batch-based gradient descent method is proved. Numerical experiments demonstrate the improved performance and computational efficiency of SpecNet2 on simulated data and image datasets.","PeriodicalId":189279,"journal":{"name":"Mathematical and Scientific Machine Learning","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-06-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126772106","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Concentration of Random Feature Matrices in High-Dimensions 高维随机特征矩阵的集中

Mathematical and Scientific Machine Learning Pub Date : 2022-04-14 DOI: 10.48550/arXiv.2204.06935

Zhijun Chen, Hayden Schaeffer, Rachel A. Ward

引用次数: 4

Error-in-variables modelling for operator learning 算子学习的变量误差建模

Mathematical and Scientific Machine Learning Pub Date : 2022-04-01 DOI: 10.48550/arXiv.2204.10909

Ravi G. Patel, Indu Manickam, Myoungkyu Lee, Mamikon A. Gulian

{"title":"Error-in-variables modelling for operator learning","authors":"Ravi G. Patel, Indu Manickam, Myoungkyu Lee, Mamikon A. Gulian","doi":"10.48550/arXiv.2204.10909","DOIUrl":"https://doi.org/10.48550/arXiv.2204.10909","url":null,"abstract":"Deep operator learning has emerged as a promising tool for reduced-order modelling and PDE model discovery. Leveraging the expressive power of deep neural networks, especially in high dimensions, such methods learn the mapping between functional state variables. While proposed methods have assumed noise only in the dependent variables, experimental and numerical data for operator learning typically exhibit noise in the independent variables as well, since both variables represent signals that are subject to measurement error. In regression on scalar data, failure to account for noisy independent variables can lead to biased parameter estimates. With noisy independent variables, linear models fitted via ordinary least squares (OLS) will show attenuation bias, wherein the slope will be underestimated. In this work, we derive an analogue of attenuation bias for linear operator regression with white noise in both the independent and dependent variables. In the nonlinear setting, we computationally demonstrate underprediction of the action of the Burgers operator in the presence of noise in the independent variable. We propose error-in-variables (EiV) models for two operator regression methods, MOR-Physics and DeepONet, and demonstrate that these new models reduce bias in the presence of noisy independent variables for a variety of operator learning problems. Considering the Burgers operator in 1D and 2D, we demonstrate that EiV operator learning robustly recovers operators in high-noise regimes that defeat OLS operator learning. We also introduce an EiV model for time-evolving PDE discovery and show that OLS and EiV perform similarly in learning the Kuramoto-Sivashinsky evolution operator from corrupted data, suggesting that the effect of bias in OLS operator learning depends on the regularity of the target operator.","PeriodicalId":189279,"journal":{"name":"Mathematical and Scientific Machine Learning","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123273371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Monte Carlo Tree Search based Hybrid Optimization of Variational Quantum Circuits 基于蒙特卡罗树搜索的变分量子电路混合优化

Mathematical and Scientific Machine Learning Pub Date : 2022-03-30 DOI: 10.48550/arXiv.2203.16707

Jiahao Yao, Haoya Li, Marin Bukov, Lin Lin, Lexing Ying

引用次数: 9

Stochastic and Private Nonconvex Outlier-Robust PCA 随机和私有非凸离群值-鲁棒PCA

Mathematical and Scientific Machine Learning Pub Date : 2022-03-17 DOI: 10.48550/arXiv.2203.09276

Tyler Maunu, Chenyun Yu, Gilad Lerman

引用次数: 2

Optimal denoising of rotationally invariant rectangular matrices 旋转不变矩形矩阵的最优去噪

Mathematical and Scientific Machine Learning Pub Date : 2022-03-15 DOI: 10.48550/arXiv.2203.07752

Emanuele Troiani, Vittorio Erba, F. Krzakala, Antoine Maillard, Lenka Zdeborov'a

引用次数: 9

Data adaptive RKHS Tikhonov regularization for learning kernels in operators 算子核学习的数据自适应RKHS Tikhonov正则化

Mathematical and Scientific Machine Learning Pub Date : 2022-03-08 DOI: 10.48550/arXiv.2203.03791

F. Lu, Quanjun Lang, Qi An

引用次数: 7

Online Weak-form Sparse Identification of Partial Differential Equations 偏微分方程的在线弱形式稀疏辨识

Mathematical and Scientific Machine Learning Pub Date : 2022-03-08 DOI: 10.48550/arXiv.2203.03979

D. Messenger, E. Dall’Anese, D. Bortz

引用次数: 9