Yihua Zhang;Prashant Khanduri;Ioannis Tsaknakis;Yuguang Yao;Mingyi Hong;Sijia Liu
{"title":"An Introduction to Bilevel Optimization: Foundations and applications in signal processing and machine learning","authors":"Yihua Zhang;Prashant Khanduri;Ioannis Tsaknakis;Yuguang Yao;Mingyi Hong;Sijia Liu","doi":"10.1109/MSP.2024.3358284","DOIUrl":"https://doi.org/10.1109/MSP.2024.3358284","url":null,"abstract":"Recently, bilevel optimization (BLO) has taken center stage in some very exciting developments in the area of signal processing (SP) and machine learning (ML). Roughly speaking, BLO is a classical optimization problem that involves two levels of hierarchy (i.e., upper and lower levels), wherein obtaining the solution to the upper-level problem requires solving the lower-level one. BLO has become popular largely because it is powerful in modeling problems in SP and ML, among others, that involve optimizing nested objective functions. Prominent applications of BLO range from resource allocation for wireless systems to adversarial ML. In this work, we focus on a class of tractable BLO problems that often appear in SP and ML applications. We provide an overview of some basic concepts of this class of BLO problems, such as their optimality conditions, standard algorithms (including their optimization principles and practical implementations) as well as how they can be leveraged to obtain state-of-the-art results for several key SP and ML applications. Further, we discuss some recent advances in BLO theory and its implications for applications, and we point out some limitations of the state of the art that require significant future research efforts. We hope that this article, together with the associated open source BLO toolbox we developed for algorithm benchmarking, can serve to accelerate the adoption of BLO as a generic tool to model, analyze, and innovate on a wide array of emerging SP and ML applications.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 1","pages":"38-59"},"PeriodicalIF":14.9,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140559307","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Learning From the Hidden Letters [President’s Message]","authors":"Min Wu","doi":"10.1109/MSP.2024.3374569","DOIUrl":"https://doi.org/10.1109/MSP.2024.3374569","url":null,"abstract":"The opportunity to write this column as the president of the IEEE Signal Processing Society (SPS) was far beyond my imagination when I first joined the SPS as a graduate student member in the 1990s. Career growth through the eyes of an SPS student member was a long journey filled with uncertainty. And at that time, SPS had few female or Asian leaders to model. Like many of you, I started by joining the tens of thousands of loyal readers of \u0000<italic>IEEE Signal Processing Magazine</i>\u0000 (\u0000<italic>SPM</i>\u0000) and became engaged as a volunteer with the SPS and IEEE in an effort to learn and grow.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 1","pages":"7-8"},"PeriodicalIF":14.9,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10502159","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140559308","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Statistical Principles of Time Reversal [Perspectives]","authors":"K. J. Ray Liu;Beibei Wang","doi":"10.1109/MSP.2023.3324472","DOIUrl":"https://doi.org/10.1109/MSP.2023.3324472","url":null,"abstract":"Time reversal is a physical principle well known for its deterministic focusing effect. Recently discovered statistical effects show that the time reversal focusing spot is not a point but has a Bessel power distribution. This finding offers accurate and reliable speed estimation indoors, where multipaths are abundant, with mostly nonline-of-sight (NLOS) conditions, and enable various indoor applications, such as wireless sensing and tracking. No known techniques can thrive in such scenarios. In essence, time reversal is an effective tool that embraces multipaths as virtual sensors with hundreds of thousands of degrees of freedom for our utilization.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 1","pages":"31-37"},"PeriodicalIF":14.9,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140559335","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"TECH RXIV: Share Your Preprint Research with the World!","authors":"","doi":"10.1109/MSP.2024.3382167","DOIUrl":"https://doi.org/10.1109/MSP.2024.3382167","url":null,"abstract":"","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 1","pages":"C3-C3"},"PeriodicalIF":14.9,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10502219","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140559278","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Signal Processing at 75: More Dynamic and Pervasive Than Ever [Perspectives]","authors":"José M. F. Moura","doi":"10.1109/MSP.2024.3367229","DOIUrl":"https://doi.org/10.1109/MSP.2024.3367229","url":null,"abstract":"The year 2023 marked the 75th anniversary of the IEEE Signal Processing Society (SPS), which was founded in 1948 as the “Professional Group on Audio” of the Institute of Radio Engineers (IRE), becoming the first IEEE Society. (The IRE, founded in 1912 with a focus on radio and then electronics, together with the American Institute of Electrical Engineers, founded in 1884 with an emphasis on power and utilities, were united in 1963 to form IEEE.) At ICASSP 2023, in Rhodes, Greece, I chaired a panel with Alex Acero of Apple, K. J. Ray Liu of Origin Wireless, Ali H. Sayed of EPFL, and Rabab K. Ward of the University of British Columbia that discussed the future of signal processing (SP) with a focus on the relations between artificial intelligence (AI) and machine learning (ML) and SP. Ours followed a panel chaired by Mos Kaveh, University of Minnesota, with Anthony Constantinides, Imperial College London, Alan Oppenheim, Massachusetts Institute of Technology, and Ron Schafer, Georgia Tech, as panelists who reminisced on the origins of SP.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 1","pages":"22-24"},"PeriodicalIF":14.9,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140559279","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Going for Sustainable Conferences [Perspectives]","authors":"Ana I. Pérez-Neira","doi":"10.1109/MSP.2024.3360955","DOIUrl":"https://doi.org/10.1109/MSP.2024.3360955","url":null,"abstract":"The research landscape is evolving very dynamically. This column reflects on it from a conference viewpoint and focuses on the importance of creating a more sustainable culture for the conference portfolio that the IEEE Signal Processing Society (SPS) offers. Among the different considerations, the role that virtual conferences can play is highlighted.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 1","pages":"25-30"},"PeriodicalIF":14.9,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140559310","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The EXPERIENCE Project: Automatic virtualization of “extended personal reality” through biomedical signal processing and explainable artificial intelligence [Applications Corner]","authors":"Gaetano Valenza;Mariano Alcañiz;Vladimir Carli;Gabriela Dudnik;Claudio Gentili;Jaime Guixeres Provinciale;Simone Rossi;Nicola Toschi;Virginie van Wassenhove","doi":"10.1109/MSP.2023.3344430","DOIUrl":"https://doi.org/10.1109/MSP.2023.3344430","url":null,"abstract":"The transformation of communication media has revolutionized social interactions, incorporating audio and video into our lives. Despite the recent availability of virtual reality (VR) technology, its widespread adoption faces obstacles. Technological challenges in creating VR environments and scientific confounding concerning interindividual variability in responses to virtual simulations are key factors hindering its broader integration. The EXPERIENCE project makes real the complex interplay among multisensory perception, emotional responses, and extended social interactions by allowing the public-at-large to create their own VR environments automatically through portable devices (e.g., smartphones/tablets) without the need for technical skills. The VR environment augmented by an individual’s physiological responses, psychological and cognitive descriptors, and behavioral outcomes defines the individual’s subjective experience, namely, an individual’s extended personal reality (EPR). The virtualization of a person’s EPR provides a holistic and quantitative environment that can be shared with others to transfer personalized psychological and emotional responses. Additionally, EPR assessment enables subsequent manipulation of the VR through explainable artificial intelligence (AI) routines merging multisensory biofeedback, individualized perception of time-space, and neuromodulation. This technology can be exploited in a plethora of innovative scenarios, including mental healthcare, gaming, e-learning, and neuroeconomics, also leading to the creation of a new market for sharing and selling (virtual) \u0000<italic>experiences</i>\u0000.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 1","pages":"60-66"},"PeriodicalIF":14.9,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140559334","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Introducing the New Area Editors for SPM [Society News]","authors":"","doi":"10.1109/MSP.2024.3377282","DOIUrl":"https://doi.org/10.1109/MSP.2024.3377282","url":null,"abstract":"Provides society information that may include news, reviews or technical notes that should be of interest to practitioners and researchers.","PeriodicalId":13246,"journal":{"name":"IEEE Signal Processing Magazine","volume":"41 1","pages":"20-21"},"PeriodicalIF":14.9,"publicationDate":"2024-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10502198","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140559336","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}