IEEE Signal Processing Magazine最新文献

{"title":"Diffusion Models for Audio Restoration: A review [Special Issue On Model-Based and Data-Driven Audio Signal Processing]","authors":"Jean-Marie Lemercier;Julius Richter;Simon Welker;Eloi Moliner;Vesa Välimäki;Timo Gerkmann","doi":"10.1109/MSP.2024.3445871","DOIUrl":"10.1109/MSP.2024.3445871","url":null,"abstract":"With the development of audio playback devices and fast data transmission, the demand for high sound quality is rising for both entertainment and communications. In this quest for better sound quality, challenges emerge from distortions and interferences originating at the recording side or caused by an imperfect transmission pipeline. To address this problem, audio restoration methods aim to recover clean sound signals from the corrupted input data. We present here audio restoration algorithms based on diffusion models, with a focus on speech enhancement and music restoration tasks. Traditional approaches, often grounded in handcrafted rules and statistical heuristics, have shaped our understanding of audio signals. In the past decades, there has been a notable shift toward data-driven methods that exploit the modeling capabilities of deep neural networks (DNNs). Deep generative models, and among them diffusion models, have emerged as powerful techniques for learning complex data distributions. However, relying solely on DNN-based learning approaches carries the risk of reducing interpretability, particularly when employing end-to-end models. Nonetheless, data-driven approaches allow more flexibility in comparison to statistical model-based frameworks, whose performance depends on distributional and statistical assumptions that can be difficult to guarantee. Here, we aim to show that diffusion models can combine the best of both worlds and offer the opportunity to design audio restoration algorithms with a good degree of interpretability and a remarkable performance in terms of sound quality. In this article, we review the use of diffusion models for audio restoration. We explain the diffusion formalism and its application to the conditional generation of clean audio signals. We believe that diffusion models open an exciting field of research with the potential to spawn new audio restoration algorithms that are natural-sounding and remain robust in difficult acoustic situations.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 6","pages":"72-84"},"PeriodicalIF":9.4,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142911946","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Calendar [Dates Ahead]","authors":"","doi":"10.1109/MSP.2024.3503133","DOIUrl":"10.1109/MSP.2024.3503133","url":null,"abstract":"","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 6","pages":"C3-C3"},"PeriodicalIF":9.4,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10819670","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142911949","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Model-Based Deep Learning for Music Information Research: Leveraging diverse knowledge sources to enhance explainability, controllability, and resource efficiency [Special Issue On Model-Based and Data-Driven Audio Signal Processing]","authors":"Gaël Richard;Vincent Lostanlen;Yi-Hsuan Yang;Meinard Müller","doi":"10.1109/MSP.2024.3415569","DOIUrl":"10.1109/MSP.2024.3415569","url":null,"abstract":"In this article, we investigate the notion of \u0000<italic>model-based deep learning</i>\u0000 in the realm of music information research (MIR). Loosely speaking, we refer to the term model-based deep learning for approaches that combine traditional knowledge-based methods with data-driven techniques, especially those based on deep learning, within a differentiable computing framework. In music, prior knowledge for instance related to sound production, music perception or music composition theory can be incorporated into the design of neural networks and associated loss functions. We outline three specific scenarios to illustrate the application of model-based deep learning in MIR, demonstrating the implementation of such concepts and their potential.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 6","pages":"51-59"},"PeriodicalIF":9.4,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142911814","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Neural Speech and Audio Coding: Modern AI technology meets traditional codecs [Special Issue On Model-Based and Data-Driven Audio Signal Processing]","authors":"Minje Kim;Jan Skoglund","doi":"10.1109/MSP.2024.3444318","DOIUrl":"10.1109/MSP.2024.3444318","url":null,"abstract":"This article explores the integration of model-based and data-driven approaches within the realm of neural speech and audio coding systems. It highlights the challenges posed by the subjective evaluation processes of speech and audio codecs and discusses the limitations of purely data-driven approaches, which often require inefficiently large architectures to match the performance of model-based methods. The study presents hybrid systems as a viable solution, offering significant improvements to the performance of conventional codecs through meticulously chosen design enhancements. Specifically, it introduces a neural network-based signal enhancer that is designed to postprocess existing codecs’ output, along with the autoencoder-based end-to-end models and LPCNet–hybrid systems that combine linear predictive coding (LPC) with neural networks. Furthermore, the article delves into predictive models that operate within custom feature spaces (TF-Codec) or predefined transform domains (MDCTNet) and examines the use of psychoacoustically calibrated loss functions to train end-to-end neural audio codecs. Through these investigations, the article demonstrates the potential of hybrid systems to advance the field of speech and audio coding by bridging the gap between traditional model-based approaches and modern data-driven techniques.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 6","pages":"85-93"},"PeriodicalIF":9.4,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142911843","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Microphone Array Signal Processing and Deep Learning for Speech Enhancement: Combining model-based and data-driven approaches to parameter estimation and filtering [Special Issue On Model-Based and Data-Driven Audio Signal Processing]","authors":"Reinhold Hëb-Umbach;Tomohiro Nakatani;Marc Delcroix;Christoph Boeddeker;Tsubasa Ochiai","doi":"10.1109/MSP.2024.3451653","DOIUrl":"10.1109/MSP.2024.3451653","url":null,"abstract":"Multichannel acoustic signal processing is a well-established and powerful tool to exploit the spatial diversity between a target signal and nontarget or noise sources for signal enhancement. However, the textbook solutions for optimal data-dependent spatial filtering rest on the knowledge of second-order statistical moments of the signals, which have traditionally been difficult to acquire. In this contribution, we compare model-based, purely data-driven, and hybrid approaches to parameter estimation and filtering, where the latter tries to combine the benefits of model-based signal processing and data-driven deep learning to overcome their individual deficiencies. We illustrate the underlying design principles with examples from noise reduction, source separation, and dereverberation.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 6","pages":"12-23"},"PeriodicalIF":9.4,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142911948","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Emerging Brain-to-Content Technologies From Generative AI and Deep Representation Learning [In the Spotlight]","authors":"Zhe Sage Chen;Xingyu Li","doi":"10.1109/MSP.2024.3484629","DOIUrl":"10.1109/MSP.2024.3484629","url":null,"abstract":"","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 6","pages":"94-104"},"PeriodicalIF":9.4,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10819698","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142911950","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Special Issue on Signal Processing for the Integrated Sensing and Communication Revolution [From the Guest Editors]","authors":"Nuria González-Prelcic;Kumar Vijay Mishra;M. R. Bhavani Shankar;Henk Wymeersch;Athina Petropulu;Pu Perry Wang","doi":"10.1109/MSP.2024.3484708","DOIUrl":"https://doi.org/10.1109/MSP.2024.3484708","url":null,"abstract":"","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 5","pages":"5-7"},"PeriodicalIF":9.4,"publicationDate":"2024-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10774073","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142761413","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Truth Is Out There: Cognitive sensing and opportunistic navigation with unknown terrestrial and nonterrestrial signals [Special Issue on Signal Processing for the Integrated Sensing and Communications Revolution]","authors":"Zaher Zak M. Kassas;Mohammad Neinavaie;Joe Khalife;Shaghayegh Shahcheraghi;Joe Saroufim","doi":"10.1109/MSP.2024.3472496","DOIUrl":"https://doi.org/10.1109/MSP.2024.3472496","url":null,"abstract":"There has been significant interest over the past few years in integrated sensing and communication (ISAC) to enable applications, such as the massive Internet of Things, highly automated transportation systems, and military surveillance. Many ISAC studies in the literature considered designing, from scratch, next-generation systems that are endowed with ISAC capabilities. This article argues that “the truth is out there” and that one can sense and exploit unknown signals, whether non-ISAC “legacy” or ISAC devised. The article presents a framework termed \u0000<italic>cognitive sensing and opportunistic navigation</i>\u0000 (\u0000<italic>COSON</i>\u0000). COSON can be thought of as an instantiation of ISAC, but, instead of having the “luxury” of designing signals with ISAC capabilities, COSON senses arbitrary, unknown communication signals and exploits them for positioning, navigation, and timing purposes. COSON is composed of four stages: 1) Blind signal acquisition, which comprises spectrum sensing and signal activity detection, blind beacon estimation, initial Doppler and Doppler rate estimation, and blind source enumeration; 2) blind signal tracking and beacon refinement; 3) interference and multipath classification; and 4) sensing and navigation. Extensive experimental results are presented, demonstrating the broad applicability of COSON to terrestrial and nonterrestrial sources, transmitting with various modulation and multiple access schemes: cellular 4G and 5G signals, GPS, and low-Earth orbit satellite (Starlink, Orbcomm, and Iridium) signals. COSON is demonstrated to localize stationary antennas and navigate unmanned aerial vehicles and a ground vehicle, to meter-level accuracy, without global navigation satellite system signals.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 5","pages":"87-99"},"PeriodicalIF":9.4,"publicationDate":"2024-12-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142761514","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Quantum State Discrimination: A Tutorial on Basic Properties of the Optimal Measurement Matrices [Lecture Notes]","authors":"Petre Stoica;Prabhu Babu;Neel Kanth Kundu","doi":"10.1109/MSP.2024.3457148","DOIUrl":"https://doi.org/10.1109/MSP.2024.3457148","url":null,"abstract":"","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 5","pages":"100-110"},"PeriodicalIF":9.4,"publicationDate":"2024-11-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10769980","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142736374","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Index Modulation for Integrated Sensing and Communications: A signal processing perspective [Special Issue on Signal Processing for the Integrated Sensing and Communications Revolution]","authors":"Ahmet M. Elbir;Abdulkadir Celik;Ahmed M. Eltawil;Moeness G. Amin","doi":"10.1109/MSP.2024.3444195","DOIUrl":"https://doi.org/10.1109/MSP.2024.3444195","url":null,"abstract":"A joint design of both sensing and communication can lead to substantial enhancement for both subsystems in terms of size and cost as well as spectrum and hardware efficiency. In the last decade, integrated sensing and communications (ISAC) has emerged as a means to efficiently utilize the spectrum on a single and shared hardware platform. Recent studies focused on developing multifunction approaches to share the spectrum between radar sensing and communications. Index modulation (IM) is one particular approach to incorporating information-bearing communication symbols into the emitted radar waveforms. While IM has been well investigated in communications-only systems, the implementation adoption of the IM concept in ISAC has recently attracted researchers to achieve improved energy/spectral efficiency while maintaining satisfactory radar sensing performance. This article focuses on recent studies on IM-ISAC and presents in detail the analytical background and relevance of the major IM-ISAC applications.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 5","pages":"44-55"},"PeriodicalIF":9.4,"publicationDate":"2024-11-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142736375","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}