训练ReLU神经网络的复杂性

IF 0.9 4区数学 Q3 MATHEMATICS, APPLIED

Discrete Optimization Pub Date : 2022-05-01 DOI:10.1016/j.disopt.2020.100620

Digvijay Boob, Santanu S. Dey, Guanghui Lan

{"title":"训练ReLU神经网络的复杂性","authors":"Digvijay Boob, Santanu S. Dey, Guanghui Lan","doi":"10.1016/j.disopt.2020.100620","DOIUrl":null,"url":null,"abstract":"<div><p>In this paper, we explore some basic questions on the complexity of training neural networks<span> with ReLU activation function. We show that it is NP-hard to train a two-hidden layer feedforward ReLU neural network. If dimension of the input data and the network topology is fixed, then we show that there exists a polynomial time algorithm for the same training problem. We also show that if sufficient over-parameterization is provided in the first hidden layer of ReLU neural network, then there is a polynomial time algorithm which finds weights such that output of the over-parameterized ReLU neural network matches with the output of the given data.</span></p></div>","PeriodicalId":50571,"journal":{"name":"Discrete Optimization","volume":"44 ","pages":"Article 100620"},"PeriodicalIF":0.9000,"publicationDate":"2022-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1016/j.disopt.2020.100620","citationCount":"50","resultStr":"{\"title\":\"Complexity of training ReLU neural network\",\"authors\":\"Digvijay Boob, Santanu S. Dey, Guanghui Lan\",\"doi\":\"10.1016/j.disopt.2020.100620\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>In this paper, we explore some basic questions on the complexity of training neural networks<span> with ReLU activation function. We show that it is NP-hard to train a two-hidden layer feedforward ReLU neural network. If dimension of the input data and the network topology is fixed, then we show that there exists a polynomial time algorithm for the same training problem. We also show that if sufficient over-parameterization is provided in the first hidden layer of ReLU neural network, then there is a polynomial time algorithm which finds weights such that output of the over-parameterized ReLU neural network matches with the output of the given data.</span></p></div>\",\"PeriodicalId\":50571,\"journal\":{\"name\":\"Discrete Optimization\",\"volume\":\"44 \",\"pages\":\"Article 100620\"},\"PeriodicalIF\":0.9000,\"publicationDate\":\"2022-05-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://sci-hub-pdf.com/10.1016/j.disopt.2020.100620\",\"citationCount\":\"50\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Discrete Optimization\",\"FirstCategoryId\":\"100\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S1572528620300542\",\"RegionNum\":4,\"RegionCategory\":\"数学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"MATHEMATICS, APPLIED\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Discrete Optimization","FirstCategoryId":"100","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1572528620300542","RegionNum":4,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"MATHEMATICS, APPLIED","Score":null,"Total":0}

引用次数: 50

摘要

本文探讨了用ReLU激活函数训练神经网络复杂性的一些基本问题。我们证明了训练一个两隐层前馈ReLU神经网络是np困难的。如果输入数据的维数和网络拓扑是固定的，那么我们证明了对于相同的训练问题存在多项式时间算法。我们还证明，如果在ReLU神经网络的第一个隐藏层提供了足够的过参数化，那么就存在一个多项式时间算法，该算法可以找到权值，使过参数化的ReLU神经网络的输出与给定数据的输出相匹配。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

Complexity of training ReLU neural network

In this paper, we explore some basic questions on the complexity of training neural networks with ReLU activation function. We show that it is NP-hard to train a two-hidden layer feedforward ReLU neural network. If dimension of the input data and the network topology is fixed, then we show that there exists a polynomial time algorithm for the same training problem. We also show that if sufficient over-parameterization is provided in the first hidden layer of ReLU neural network, then there is a polynomial time algorithm which finds weights such that output of the over-parameterized ReLU neural network matches with the output of the given data.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Discrete Optimization 管理科学-应用数学

CiteScore

2.10

自引率

9.10%

发文量

审稿时长

>12 weeks

期刊介绍： Discrete Optimization publishes research papers on the mathematical, computational and applied aspects of all areas of integer programming and combinatorial optimization. In addition to reports on mathematical results pertinent to discrete optimization, the journal welcomes submissions on algorithmic developments, computational experiments, and novel applications (in particular, large-scale and real-time applications). The journal also publishes clearly labelled surveys, reviews, short notes, and open problems. Manuscripts submitted for possible publication to Discrete Optimization should report on original research, should not have been previously published, and should not be under consideration for publication by any other journal.