Peng Xue;Min Gan;Fang Yuan;Guang-Yong Chen;C. L. Philip Chen
{"title":"基于移动平均的可分离非线性问题变量投影","authors":"Peng Xue;Min Gan;Fang Yuan;Guang-Yong Chen;C. L. Philip Chen","doi":"10.1109/LSP.2025.3563133","DOIUrl":null,"url":null,"abstract":"The identification of separable nonlinear models, prevalent in tasks such as signal analysis, image processing, time series analysis, and machine learning, presents a non-convex optimization challenge that necessitates the development of efficient identification algorithms. The Variable Projection (VP) algorithm has been proven to be quite effective for addressing these problems; however, traditional VP relying on the Hessian matrix and its inverse are highly time-consuming and unsuitable for complex, large-scale applications. This letter introduces a novel approach that employs the exponential moving average of gradient and gradient estimation bias to indirectly estimate the curvature of the objective landscape, proposing a Moving Average-based Variable Projection method (MAVP). The proposed algorithm utilizes only gradient information and can properly tackle the coupling relationships between different parameters during the optimization process, thereby achieving faster convergence. Numerical results on nonlinear time series analysis and image reconstruction demonstrate that the MAVP algorithm exhibits significant efficiency and effectiveness.","PeriodicalId":13154,"journal":{"name":"IEEE Signal Processing Letters","volume":"32 ","pages":"1900-1904"},"PeriodicalIF":3.2000,"publicationDate":"2025-04-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Moving Average-Based Variable Projection for Separable Nonlinear Problems\",\"authors\":\"Peng Xue;Min Gan;Fang Yuan;Guang-Yong Chen;C. L. Philip Chen\",\"doi\":\"10.1109/LSP.2025.3563133\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"The identification of separable nonlinear models, prevalent in tasks such as signal analysis, image processing, time series analysis, and machine learning, presents a non-convex optimization challenge that necessitates the development of efficient identification algorithms. The Variable Projection (VP) algorithm has been proven to be quite effective for addressing these problems; however, traditional VP relying on the Hessian matrix and its inverse are highly time-consuming and unsuitable for complex, large-scale applications. This letter introduces a novel approach that employs the exponential moving average of gradient and gradient estimation bias to indirectly estimate the curvature of the objective landscape, proposing a Moving Average-based Variable Projection method (MAVP). The proposed algorithm utilizes only gradient information and can properly tackle the coupling relationships between different parameters during the optimization process, thereby achieving faster convergence. Numerical results on nonlinear time series analysis and image reconstruction demonstrate that the MAVP algorithm exhibits significant efficiency and effectiveness.\",\"PeriodicalId\":13154,\"journal\":{\"name\":\"IEEE Signal Processing Letters\",\"volume\":\"32 \",\"pages\":\"1900-1904\"},\"PeriodicalIF\":3.2000,\"publicationDate\":\"2025-04-21\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Signal Processing Letters\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10971927/\",\"RegionNum\":2,\"RegionCategory\":\"工程技术\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Signal Processing Letters","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/10971927/","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
Moving Average-Based Variable Projection for Separable Nonlinear Problems
The identification of separable nonlinear models, prevalent in tasks such as signal analysis, image processing, time series analysis, and machine learning, presents a non-convex optimization challenge that necessitates the development of efficient identification algorithms. The Variable Projection (VP) algorithm has been proven to be quite effective for addressing these problems; however, traditional VP relying on the Hessian matrix and its inverse are highly time-consuming and unsuitable for complex, large-scale applications. This letter introduces a novel approach that employs the exponential moving average of gradient and gradient estimation bias to indirectly estimate the curvature of the objective landscape, proposing a Moving Average-based Variable Projection method (MAVP). The proposed algorithm utilizes only gradient information and can properly tackle the coupling relationships between different parameters during the optimization process, thereby achieving faster convergence. Numerical results on nonlinear time series analysis and image reconstruction demonstrate that the MAVP algorithm exhibits significant efficiency and effectiveness.
期刊介绍:
The IEEE Signal Processing Letters is a monthly, archival publication designed to provide rapid dissemination of original, cutting-edge ideas and timely, significant contributions in signal, image, speech, language and audio processing. Papers published in the Letters can be presented within one year of their appearance in signal processing conferences such as ICASSP, GlobalSIP and ICIP, and also in several workshop organized by the Signal Processing Society.