Ekaterina Trimbach, Edward Duc Hien Nguyen, César A Uribe
{"title":"On Acceleration of Gradient-Based Empirical Risk Minimization using Local Polynomial Regression.","authors":"Ekaterina Trimbach, Edward Duc Hien Nguyen, César A Uribe","doi":"10.23919/ecc55457.2022.9838261","DOIUrl":"10.23919/ecc55457.2022.9838261","url":null,"abstract":"<p><p>We study the acceleration of the Local Polynomial Interpolation-based Gradient Descent method (LPI-GD) recently proposed for the approximate solution of empirical risk minimization problems (ERM). We focus on loss functions that are strongly convex and smooth with condition number <i>σ</i>. We additionally assume the loss function is <i>η</i>-Hölder continuous with respect to the data. The oracle complexity of LPI-GD is <math> <mrow><mover><mi>O</mi> <mo>˜</mo></mover> <mrow><mo>(</mo> <mrow><mi>σ</mi> <msup><mi>m</mi> <mi>d</mi></msup> <mspace></mspace> <mtext>log</mtext> <mo>(</mo> <mn>1</mn> <mo>/</mo> <mi>ε</mi> <mo>)</mo></mrow> <mo>)</mo></mrow> </mrow> </math> for a desired accuracy <i>ε</i>, where <i>d</i> is the dimension of the parameter space, and <i>m</i> is the cardinality of an approximation grid. The factor <i>m</i> <sup><i>d</i></sup> can be shown to scale as <i>O</i>((1/<i>ε</i>) <sup><i>d</i>/2<i>η</i></sup> ). LPI-GD has been shown to have better oracle complexity than gradient descent (GD) and stochastic gradient descent (SGD) for certain parameter regimes. We propose two accelerated methods for the ERM problem based on LPI-GD and show an oracle complexity of <math> <mrow><mover><mi>O</mi> <mo>˜</mo></mover> <mrow><mo>(</mo> <mrow><msqrt><mi>σ</mi></msqrt> <msup><mi>m</mi> <mi>d</mi></msup> <mspace></mspace> <mtext>log</mtext> <mo>(</mo> <mn>1</mn> <mo>/</mo> <mi>ε</mi> <mo>)</mo></mrow> <mo>)</mo></mrow> </mrow> </math> . Moreover, we provide the first empirical study on local polynomial interpolation-based gradient methods and corroborate that LPI-GD has better performance than GD and SGD in some scenarios, and the proposed methods achieve acceleration.</p>","PeriodicalId":72704,"journal":{"name":"Control Conference (ECC) ... European. European Control Conference","volume":"2022 ","pages":"429-434"},"PeriodicalIF":0.0,"publicationDate":"2022-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9581727/pdf/nihms-1842409.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9942992","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Model-Based Estimation of Wheel Slip in Locomotives","authors":"C. V. V. D. Merwe, J. D. L. Roux","doi":"10.23919/ECC55457.2022.9838075","DOIUrl":"https://doi.org/10.23919/ECC55457.2022.9838075","url":null,"abstract":"","PeriodicalId":72704,"journal":{"name":"Control Conference (ECC) ... European. European Control Conference","volume":"1 1","pages":"2124-2129"},"PeriodicalIF":0.0,"publicationDate":"2022-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"79948710","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Computationally efficient application of Sequential Monte Carlo expectation maximization to confined single particle tracking.","authors":"Ye Lin, Sean B Andersson","doi":"10.23919/ecc54610.2021.9655194","DOIUrl":"https://doi.org/10.23919/ecc54610.2021.9655194","url":null,"abstract":"<p><p>Single Particle Tracking (SPT) plays a crucial role in biophysics through its ability to reveal dynamic mechanisms and physical properties of biological macromolecules moving inside living cells. Such molecules are often subject to confinement and important information can be revealed by understanding the mobility of the molecules and the size of the domain they are restricted to. In previous work, we introduced a method known as Sequential Monte Carlo-Expectation Maximization (SMC-EM) to simultaneously estimate particle trajectories and model parameters. In this paper, we describe three modifications to SMC-EM aimed at improving its computationally efficiency and demonstrate it through analysis of simulated SPT data of a particle in a three dimensional confined environment. The first two modifications use approximation methods to reduce the complexity of the original motion and measurement models without significant loss of accuracy. The third modification replaces the previous SMC methods with a Gaussian particle filter combined with a backward simulation particle smoother, trading off some level of generality for improved computational performance. In addition, we take advantage of the improved efficiency to investigate the effect of data length on performance in localization and parameter estimation.</p>","PeriodicalId":72704,"journal":{"name":"Control Conference (ECC) ... European. European Control Conference","volume":"2021 ","pages":"1919-1924"},"PeriodicalIF":0.0,"publicationDate":"2021-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8785855/pdf/nihms-1724592.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"39859509","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Stepanov, V. Musatov, I. Egorov, S. Pchelintzeva, A. Stepanov, O. Stepanova, A. R. Berkaev, A. Ishanov, A. Nenashev, A. P. Nijazov
{"title":"Control of Mobile Plant with use of Interface Brain Computer","authors":"M. Stepanov, V. Musatov, I. Egorov, S. Pchelintzeva, A. Stepanov, O. Stepanova, A. R. Berkaev, A. Ishanov, A. Nenashev, A. P. Nijazov","doi":"10.23919/ECC51009.2020.9144000","DOIUrl":"https://doi.org/10.23919/ECC51009.2020.9144000","url":null,"abstract":"","PeriodicalId":72704,"journal":{"name":"Control Conference (ECC) ... European. European Control Conference","volume":"69 1","pages":"290-293"},"PeriodicalIF":0.0,"publicationDate":"2020-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"76527412","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
P. Bevilacqua, Marco Frego, D. Fontanelli, L. Palopoli
{"title":"A novel formalisation of the Markov-Dubins problem","authors":"P. Bevilacqua, Marco Frego, D. Fontanelli, L. Palopoli","doi":"10.23919/ECC51009.2020.9143597","DOIUrl":"https://doi.org/10.23919/ECC51009.2020.9143597","url":null,"abstract":"","PeriodicalId":72704,"journal":{"name":"Control Conference (ECC) ... European. European Control Conference","volume":"19 1","pages":"1987-1992"},"PeriodicalIF":0.0,"publicationDate":"2020-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77902285","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Boris I Godoy, Nicholas A Vickers, Y Lin, Sean B Andersson
{"title":"Estimation of general time-varying single particle tracking linear models using local likelihood.","authors":"Boris I Godoy, Nicholas A Vickers, Y Lin, Sean B Andersson","doi":"10.23919/ecc51009.2020.9143818","DOIUrl":"10.23919/ecc51009.2020.9143818","url":null,"abstract":"<p><p>In this work, we study a general approach to the estimation of single particle tracking models with time-varying parameters. The main idea is to use local Maximum Likelihood (ML), applying a sliding window over the data and estimating the model parameters in each window. We combine local ML with Expectation Maximization to iteratively find the ML estimate in each window, an approach that is amenable to generalization to nonlinear models. Results using controlled-experimental data generated in our lab show that our proposed algorithm is able to track changes in the parameters as they evolve during a trajectory under real-world experimental conditions, outperforming other algorithms of similar nature.</p>","PeriodicalId":72704,"journal":{"name":"Control Conference (ECC) ... European. European Control Conference","volume":"2020 ","pages":"527-533"},"PeriodicalIF":0.0,"publicationDate":"2020-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8411989/pdf/nihms-1611711.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"39387490","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Stepanov, V. Musatov, I. Egorov, S. Pchelintzeva, A. Stepanov, O. Stepanova, A. R. Berkaev, A. Ishanov, A. Nenashev, A. P. Nijazov
{"title":"Subsystem of decision making support of robotics hardware-software","authors":"M. Stepanov, V. Musatov, I. Egorov, S. Pchelintzeva, A. Stepanov, O. Stepanova, A. R. Berkaev, A. Ishanov, A. Nenashev, A. P. Nijazov","doi":"10.23919/ECC51009.2020.9143938","DOIUrl":"https://doi.org/10.23919/ECC51009.2020.9143938","url":null,"abstract":"","PeriodicalId":72704,"journal":{"name":"Control Conference (ECC) ... European. European Control Conference","volume":"57 1","pages":"1069-1072"},"PeriodicalIF":0.0,"publicationDate":"2020-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72916903","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
S. Konstantinov, A. Diveev, Elena A. Sofronova, I. Zelinka
{"title":"Optimal Control System Synthesis Based on the Approximation of Extremals by Symbolic Regression","authors":"S. Konstantinov, A. Diveev, Elena A. Sofronova, I. Zelinka","doi":"10.23919/ECC51009.2020.9143798","DOIUrl":"https://doi.org/10.23919/ECC51009.2020.9143798","url":null,"abstract":"","PeriodicalId":72704,"journal":{"name":"Control Conference (ECC) ... European. European Control Conference","volume":"165 1","pages":"2021-2026"},"PeriodicalIF":0.0,"publicationDate":"2020-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"73658161","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Least squares realization of LTI models is an eigenvalue problem","authors":"B. Moor","doi":"10.23919/ECC.2019.8795987","DOIUrl":"https://doi.org/10.23919/ECC.2019.8795987","url":null,"abstract":"We show how least squares optimal realization of autonomous linear time-invariant dynamical systems from given data, reduces to the solution of an eigenvalue problem. In this short paper, we can only schematically sketch the different steps: The first order optimality conditions result in a multi-parameter eigenvalue problem. The eigenvalue $n$ -tuples are calculated from the null space of a quasi-Toeplitz block Macaulay matrix, which is shown to be multishift-invariant. This last property is then exploited via nD ‘exact’ realization theory, leading through several eigenvalue problems to the optimal model parameters.","PeriodicalId":72704,"journal":{"name":"Control Conference (ECC) ... European. European Control Conference","volume":"64 1","pages":"2270-2275"},"PeriodicalIF":0.0,"publicationDate":"2019-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"74468592","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Dynamic extension for direct integrability of singular solutions in optimal control problems","authors":"P. D. Giamberardino","doi":"10.23919/ECC.2019.8795907","DOIUrl":"https://doi.org/10.23919/ECC.2019.8795907","url":null,"abstract":"The paper addresses the problem of optimal control design in presence of singular solutions. For this case, a procedure for avoiding the integration of the costate dynamics is proposed, giving the conditions under which the costate can be directly computed, under controllability condition for the dynamics, and presenting an approach for extending this property by a dynamic extension. The procedure is here described for a single input systems and for the case in which the first step of the iterative procedure is sufficient to get the solution. An example is used to show the feasibility of the approach.","PeriodicalId":72704,"journal":{"name":"Control Conference (ECC) ... European. European Control Conference","volume":"20 1","pages":"4216-4221"},"PeriodicalIF":0.0,"publicationDate":"2019-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"83825810","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}