蝴蝶稀疏矩阵分解的高效识别

IF 1.9 Q1 MATHEMATICS, APPLIED
Léon Zheng, E. Riccietti, R. Gribonval
{"title":"蝴蝶稀疏矩阵分解的高效识别","authors":"Léon Zheng, E. Riccietti, R. Gribonval","doi":"10.1137/22m1488727","DOIUrl":null,"url":null,"abstract":"Fast transforms correspond to factorizations of the form $\\mathbf{Z} = \\mathbf{X}^{(1)} \\ldots \\mathbf{X}^{(J)}$, where each factor $ \\mathbf{X}^{(\\ell)}$ is sparse and possibly structured. This paper investigates essential uniqueness of such factorizations, i.e., uniqueness up to unavoidable scaling ambiguities. Our main contribution is to prove that any $N \\times N$ matrix having the so-called butterfly structure admits an essentially unique factorization into $J$ butterfly factors (where $N = 2^{J}$), and that the factors can be recovered by a hierarchical factorization method, which consists in recursively factorizing the considered matrix into two factors. This hierarchical identifiability property relies on a simple identifiability condition in the two-layer and fixed-support setting. This approach contrasts with existing ones that fit the product of butterfly factors to a given matrix via gradient descent. The proposed method can be applied in particular to retrieve the factorization of the Hadamard or the discrete Fourier transform matrices of size $N=2^J$. Computing such factorizations costs $\\mathcal{O}(N^{2})$, which is of the order of dense matrix-vector multiplication, while the obtained factorizations enable fast $\\mathcal{O}(N \\log N)$ matrix-vector multiplications and have the potential to be applied to compress deep neural networks.","PeriodicalId":74797,"journal":{"name":"SIAM journal on mathematics of data science","volume":"50 1","pages":"22-49"},"PeriodicalIF":1.9000,"publicationDate":"2021-10-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"4","resultStr":"{\"title\":\"Efficient Identification of Butterfly Sparse Matrix Factorizations\",\"authors\":\"Léon Zheng, E. Riccietti, R. Gribonval\",\"doi\":\"10.1137/22m1488727\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Fast transforms correspond to factorizations of the form $\\\\mathbf{Z} = \\\\mathbf{X}^{(1)} \\\\ldots \\\\mathbf{X}^{(J)}$, where each factor $ \\\\mathbf{X}^{(\\\\ell)}$ is sparse and possibly structured. This paper investigates essential uniqueness of such factorizations, i.e., uniqueness up to unavoidable scaling ambiguities. Our main contribution is to prove that any $N \\\\times N$ matrix having the so-called butterfly structure admits an essentially unique factorization into $J$ butterfly factors (where $N = 2^{J}$), and that the factors can be recovered by a hierarchical factorization method, which consists in recursively factorizing the considered matrix into two factors. This hierarchical identifiability property relies on a simple identifiability condition in the two-layer and fixed-support setting. This approach contrasts with existing ones that fit the product of butterfly factors to a given matrix via gradient descent. The proposed method can be applied in particular to retrieve the factorization of the Hadamard or the discrete Fourier transform matrices of size $N=2^J$. Computing such factorizations costs $\\\\mathcal{O}(N^{2})$, which is of the order of dense matrix-vector multiplication, while the obtained factorizations enable fast $\\\\mathcal{O}(N \\\\log N)$ matrix-vector multiplications and have the potential to be applied to compress deep neural networks.\",\"PeriodicalId\":74797,\"journal\":{\"name\":\"SIAM journal on mathematics of data science\",\"volume\":\"50 1\",\"pages\":\"22-49\"},\"PeriodicalIF\":1.9000,\"publicationDate\":\"2021-10-04\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"4\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"SIAM journal on mathematics of data science\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1137/22m1488727\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"MATHEMATICS, APPLIED\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"SIAM journal on mathematics of data science","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1137/22m1488727","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MATHEMATICS, APPLIED","Score":null,"Total":0}
引用次数: 4

摘要

快速变换对应于$\mathbf{Z} = \mathbf{X}^{(1)} \ldots \mathbf{X}^{(J)}$的分解形式,其中每个因子$\mathbf{X}^{(\ well)}$是稀疏的并且可能是结构化的。本文研究了这种分解的本质唯一性,即唯一性到不可避免的尺度歧义。我们的主要贡献是证明了任何具有所谓蝴蝶结构的$N \ * N$矩阵都可以被唯一地分解为$J$蝴蝶因子(其中$N = 2^{J}$),并且这些因子可以通过分层分解方法恢复,该方法包括将所考虑的矩阵递归分解为两个因子。这种分层的可识别属性依赖于两层固定支持设置中的一个简单的可识别条件。这种方法与现有的通过梯度下降将蝴蝶因子的乘积拟合到给定矩阵的方法形成了对比。该方法特别适用于检索大小为$N=2^J$的Hadamard或离散傅里叶变换矩阵的因式分解。计算这样的因数分解花费$\mathcal{O}(N^{2})$,这是密集矩阵-向量乘法的顺序,而获得的因数分解实现了快速的$\mathcal{O}(N \log N)$矩阵-向量乘法,并且具有应用于压缩深度神经网络的潜力。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Efficient Identification of Butterfly Sparse Matrix Factorizations
Fast transforms correspond to factorizations of the form $\mathbf{Z} = \mathbf{X}^{(1)} \ldots \mathbf{X}^{(J)}$, where each factor $ \mathbf{X}^{(\ell)}$ is sparse and possibly structured. This paper investigates essential uniqueness of such factorizations, i.e., uniqueness up to unavoidable scaling ambiguities. Our main contribution is to prove that any $N \times N$ matrix having the so-called butterfly structure admits an essentially unique factorization into $J$ butterfly factors (where $N = 2^{J}$), and that the factors can be recovered by a hierarchical factorization method, which consists in recursively factorizing the considered matrix into two factors. This hierarchical identifiability property relies on a simple identifiability condition in the two-layer and fixed-support setting. This approach contrasts with existing ones that fit the product of butterfly factors to a given matrix via gradient descent. The proposed method can be applied in particular to retrieve the factorization of the Hadamard or the discrete Fourier transform matrices of size $N=2^J$. Computing such factorizations costs $\mathcal{O}(N^{2})$, which is of the order of dense matrix-vector multiplication, while the obtained factorizations enable fast $\mathcal{O}(N \log N)$ matrix-vector multiplications and have the potential to be applied to compress deep neural networks.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信