关于与规模无关的 ReLU 网络样本复杂性

IF 0.7 4区计算机科学 Q4 COMPUTER SCIENCE, INFORMATION SYSTEMS

Information Processing Letters Pub Date : 2024-02-07 DOI:10.1016/j.ipl.2024.106482

Mark Sellke

{"title":"关于与规模无关的 ReLU 网络样本复杂性","authors":"Mark Sellke","doi":"10.1016/j.ipl.2024.106482","DOIUrl":null,"url":null,"abstract":"<div><p>We study the sample complexity of learning ReLU neural networks from the point of view of generalization. Given norm constraints on the weight matrices, a common approach is to estimate the Rademacher complexity of the associated function class. Previously <span>[9]</span> obtained a bound independent of the network size (scaling with a product of Frobenius norms) except for a factor of the square-root depth. We give a refinement which often has no explicit depth-dependence at all.</p></div>","PeriodicalId":56290,"journal":{"name":"Information Processing Letters","volume":"186 ","pages":"Article 106482"},"PeriodicalIF":0.7000,"publicationDate":"2024-02-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"On size-independent sample complexity of ReLU networks\",\"authors\":\"Mark Sellke\",\"doi\":\"10.1016/j.ipl.2024.106482\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><p>We study the sample complexity of learning ReLU neural networks from the point of view of generalization. Given norm constraints on the weight matrices, a common approach is to estimate the Rademacher complexity of the associated function class. Previously <span>[9]</span> obtained a bound independent of the network size (scaling with a product of Frobenius norms) except for a factor of the square-root depth. We give a refinement which often has no explicit depth-dependence at all.</p></div>\",\"PeriodicalId\":56290,\"journal\":{\"name\":\"Information Processing Letters\",\"volume\":\"186 \",\"pages\":\"Article 106482\"},\"PeriodicalIF\":0.7000,\"publicationDate\":\"2024-02-07\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Information Processing Letters\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0020019024000127\",\"RegionNum\":4,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q4\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Information Processing Letters","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0020019024000127","RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}

引用次数: 0

摘要

我们从泛化的角度研究学习 ReLU 神经网络的样本复杂性。考虑到权重矩阵的规范约束，一种常见的方法是估算相关函数类的拉德马赫复杂度。在此之前 [9] 的研究中，除了平方根深度因子之外，我们得到了一个与网络规模无关的约束条件（与 Frobenius 准则的乘积成比例）。我们给出的改进方法通常完全不存在明确的深度依赖性。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

查看原文本刊更多论文

On size-independent sample complexity of ReLU networks

We study the sample complexity of learning ReLU neural networks from the point of view of generalization. Given norm constraints on the weight matrices, a common approach is to estimate the Rademacher complexity of the associated function class. Previously [9] obtained a bound independent of the network size (scaling with a product of Frobenius norms) except for a factor of the square-root depth. We give a refinement which often has no explicit depth-dependence at all.

求助全文

通过发布文献求助，成功后即可免费获取论文全文。去求助

来源期刊

Information Processing Letters 工程技术-计算机：信息系统

CiteScore

1.80

自引率

0.00%

发文量

审稿时长

7.3 months

期刊介绍： Information Processing Letters invites submission of original research articles that focus on fundamental aspects of information processing and computing. This naturally includes work in the broadly understood field of theoretical computer science; although papers in all areas of scientific inquiry will be given consideration, provided that they describe research contributions credibly motivated by applications to computing and involve rigorous methodology. High quality experimental papers that address topics of sufficiently broad interest may also be considered. Since its inception in 1971, Information Processing Letters has served as a forum for timely dissemination of short, concise and focused research contributions. Continuing with this tradition, and to expedite the reviewing process, manuscripts are generally limited in length to nine pages when they appear in print.