IEEE Signal Processing Magazine最新文献

Rapture of the Deep: Highs and lows of sparsity in a world of depths 深渊的狂喜：深渊世界中稀疏的高低

IF 9.6 1区工程技术

IEEE Signal Processing Magazine Pub Date : 2026-03-01 Epub Date: 2026-04-13 DOI: 10.1109/MSP.2025.3611564

Rémi Gribonval;Elisa Riccietti;Quoc-Tung Le;Léon Zheng

{"title":"Rapture of the Deep: Highs and lows of sparsity in a world of depths","authors":"Rémi Gribonval;Elisa Riccietti;Quoc-Tung Le;Léon Zheng","doi":"10.1109/MSP.2025.3611564","DOIUrl":"https://doi.org/10.1109/MSP.2025.3611564","url":null,"abstract":"Promoting sparsity in deep networks is a natural way to control their complexity, and it is a timely endeavor since practical neural model sizes have grown to unprecedented levels. The lessons from sparsity in linear inverse problems also bear the promise of many other benefits beyond such computational aspects, from statistical significance to explainability. Can these promises be fulfilled? Can we safely leverage the know-how of sparsity-promoting regularizers for inverse problems to harness sparsity in deeper contexts, linear or not? This article surveys the curses and blessings of deep sparsity. After a reminder on the main lessons from inverse problems, we tour a number of results that challenge their immediate deep extensions, from both a mathematical and a computational perspective. In particular, we highlight that <inline-formula><tex-math>${mathit{ell}}^{1}$</tex-math></inline-formula> regularization does not always lead to sparsity, and that optimization with a prescribed set of allowed nonzero coefficients can be NP-hard. We emphasize the role of rescaling invariances in these phenomena and the need to favor structured sparsity to keep sparse network training problems under control, ensure their stability, and actually enable efficient network implementations on GPUs. We finally outline the promises and challenges of a flexible family of <italic>Kronecker sparsity structures</i>, which extend the classical butterfly structure and appear in many classical scientific computing applications and that have also recently emerged in deep learning.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"43 2","pages":"10-23"},"PeriodicalIF":9.6,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147665344","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

SPS Advance Your Career SPS促进你的职业发展

IF 9.6 1区工程技术

IEEE Signal Processing Magazine Pub Date : 2026-03-01 Epub Date: 2026-04-13 DOI: 10.1109/MSP.2026.3676841

引用次数: 0

SPS Podcast SPS播客

IF 9.6 1区工程技术

IEEE Signal Processing Magazine Pub Date : 2026-03-01 Epub Date: 2026-04-13 DOI: 10.1109/MSP.2026.3676840

引用次数: 0

Probabilistic Graphical Models Meet Deep Learning for Semantic Segmentation: Mathematical connections and recent developments 概率图模型满足语义分割的深度学习：数学联系和最新发展

IF 9.6 1区工程技术

IEEE Signal Processing Magazine Pub Date : 2026-03-01 Epub Date: 2026-04-13 DOI: 10.1109/MSP.2025.3648958

Martina Pastorino;Gabriele Moser;Sebastiano B. Serpico;Josiane Zerubia

引用次数: 0

The Nasreddin Hodja Principle and the Mathematics of Deep Learning [From the Editor] Nasreddin Hodja原理与深度学习数学

IF 9.6 1区工程技术

IEEE Signal Processing Magazine Pub Date : 2026-03-01 Epub Date: 2026-04-13 DOI: 10.1109/MSP.2026.3676148

Tülay Adali

引用次数: 0

Continual Learning Through the Lens of Adaptive Filtering: A mathematical tutorial 通过自适应滤波镜头的持续学习：数学教程

IF 9.6 1区工程技术

IEEE Signal Processing Magazine Pub Date : 2026-03-01 Epub Date: 2026-04-13 DOI: 10.1109/MSP.2026.3657516

Liangzu Peng;René Vidal

引用次数: 0

IEEE Feedback IEEE反馈

IF 9.6 1区工程技术

IEEE Signal Processing Magazine Pub Date : 2026-03-01 Epub Date: 2026-04-13 DOI: 10.1109/MSP.2026.3677929

引用次数: 0

ICIP ICIP

IF 9.6 1区工程技术

IEEE Signal Processing Magazine Pub Date : 2026-03-01 Epub Date: 2026-04-13 DOI: 10.1109/MSP.2026.3678843

引用次数: 0

Flow-Based Generative Models as Iterative Algorithms in Probability Space: An intuitive mathematical framework [Special Issue on the Mathematics of Deep Learning] 基于流的生成模型作为概率空间中的迭代算法：一个直观的数学框架[深度学习数学专刊]

IF 9.6 1区工程技术

IEEE Signal Processing Magazine Pub Date : 2026-03-01 Epub Date: 2026-04-13 DOI: 10.1109/MSP.2025.3609527

Yao Xie;Xiuyuan Cheng

{"title":"Flow-Based Generative Models as Iterative Algorithms in Probability Space: An intuitive mathematical framework [Special Issue on the Mathematics of Deep Learning]","authors":"Yao Xie;Xiuyuan Cheng","doi":"10.1109/MSP.2025.3609527","DOIUrl":"https://doi.org/10.1109/MSP.2025.3609527","url":null,"abstract":"Flow-based generative models have emerged as a powerful class of deep generative models, offering exact likelihood estimation, invertible sample transformations, and reliable and efficient sampling, making them particularly well-suited for applications in signal processing, anomaly detection, and structured data synthesis. Unlike diffusion models, which rely on stochastic differential equations (SDEs) for progressive denoising, flow-based models define deterministic transformations governed by ordinary differential equations (ODEs), allowing for faster inference and interpretable probabilistic modeling. This tutorial presents a rigorous mathematical framework for flow-based generative models, positioning them as iterative algorithms in probability space and exploring their connections to optimal transport and Wasserstein gradient flows. We discuss key algorithmic insights, including continuous normalizing flows (CNFs), flow-matching (FM), and distributionally robust optimization (DRO), which enable efficient, high-dimensional generative modeling with theoretical guarantees. We also examine the role of flow-based models in progressive training schemes, demonstrating their convergence properties and generative guarantees under structured optimization frameworks. By bridging mathematical theory with practical implementation, this tutorial aims to provide researchers and practitioners with a comprehensive foundation in flow-based generative modeling and its applications in signal processing and beyond.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"43 2","pages":"37-50"},"PeriodicalIF":9.6,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147665512","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models [Special Issue on the Mathematics of Deep Learning] 深度学习的随机矩阵理论：超越线性模型的特征值[深度学习数学专刊]

IF 9.6 1区工程技术

IEEE Signal Processing Magazine Pub Date : 2026-03-01 Epub Date: 2026-04-14 DOI: 10.1109/MSP.2025.3618012

Zhenyu Liao;Michael W. Mahoney

{"title":"Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models [Special Issue on the Mathematics of Deep Learning]","authors":"Zhenyu Liao;Michael W. Mahoney","doi":"10.1109/MSP.2025.3618012","DOIUrl":"https://doi.org/10.1109/MSP.2025.3618012","url":null,"abstract":"Modern machine learning (ML) and deep neural networks (DNNs) often operate on high-dimensional data and rely on overparameterized models, where classical low-dimensional intuitions break down. In particular, the proportional regime where the data dimension, sample size, and number of model parameters are all large and comparable, gives rise to novel and sometimes counterintuitive behaviors. This article extends traditional random matrix theory (RMT) beyond eigenvalue-based analysis of linear models to address the challenges posed by nonlinear ML models such as DNNs in this regime. We introduce the concept of high-dimensional equivalent, which unifies and generalizes both deterministic equivalent and linear equivalent, to systematically address three technical challenges: high dimensionality, nonlinearity, and the need to analyze generic eigenspectral functionals. Leveraging this framework, we provide precise characterizations of the training and generalization performance of linear models, nonlinear shallow networks, and deep networks. Our results capture rich phenomena, including scaling laws, double descent, and nonlinear learning dynamics, offering a unified perspective on the theoretical understanding of deep learning in high dimensions.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"43 2","pages":"93-106"},"PeriodicalIF":9.6,"publicationDate":"2026-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147696206","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0