Complexity of training ReLU neural network

IF 0.9 4区数学 Q3 MATHEMATICS, APPLIED

Discrete Optimization Pub Date : 2022-05-01 DOI:10.1016/j.disopt.2020.100620

Digvijay Boob, Santanu S. Dey, Guanghui Lan

引用次数: 50

Abstract

In this paper, we explore some basic questions on the complexity of training neural networks with ReLU activation function. We show that it is NP-hard to train a two-hidden layer feedforward ReLU neural network. If dimension of the input data and the network topology is fixed, then we show that there exists a polynomial time algorithm for the same training problem. We also show that if sufficient over-parameterization is provided in the first hidden layer of ReLU neural network, then there is a polynomial time algorithm which finds weights such that output of the over-parameterized ReLU neural network matches with the output of the given data.

查看原文本刊更多论文

训练ReLU神经网络的复杂性

本文探讨了用ReLU激活函数训练神经网络复杂性的一些基本问题。我们证明了训练一个两隐层前馈ReLU神经网络是np困难的。如果输入数据的维数和网络拓扑是固定的，那么我们证明了对于相同的训练问题存在多项式时间算法。我们还证明，如果在ReLU神经网络的第一个隐藏层提供了足够的过参数化，那么就存在一个多项式时间算法，该算法可以找到权值，使过参数化的ReLU神经网络的输出与给定数据的输出相匹配。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Discrete Optimization 管理科学-应用数学

CiteScore

2.10

自引率

9.10%

发文量

审稿时长

>12 weeks

期刊介绍： Discrete Optimization publishes research papers on the mathematical, computational and applied aspects of all areas of integer programming and combinatorial optimization. In addition to reports on mathematical results pertinent to discrete optimization, the journal welcomes submissions on algorithmic developments, computational experiments, and novel applications (in particular, large-scale and real-time applications). The journal also publishes clearly labelled surveys, reviews, short notes, and open problems. Manuscripts submitted for possible publication to Discrete Optimization should report on original research, should not have been previously published, and should not be under consideration for publication by any other journal.