Statistics and Computing最新文献_第8页

Stochastic three-term conjugate gradient method with variance technique for non-convex learning 用于非凸学习的随机三项共轭梯度法与方差技术

IF 2.2 2区数学

Statistics and Computing Pub Date : 2024-03-27 DOI: 10.1007/s11222-024-10409-5

Chen Ouyang, Chenkaixiang Lu, Xiong Zhao, Ruping Huang, Gonglin Yuan, Yiyan Jiang

{"title":"Stochastic three-term conjugate gradient method with variance technique for non-convex learning","authors":"Chen Ouyang, Chenkaixiang Lu, Xiong Zhao, Ruping Huang, Gonglin Yuan, Yiyan Jiang","doi":"10.1007/s11222-024-10409-5","DOIUrl":"https://doi.org/10.1007/s11222-024-10409-5","url":null,"abstract":"In the training process of machine learning, the minimization of the empirical risk loss function is often used to measure the difference between the model’s predicted value and the real value. Stochastic gradient descent is very popular for this type of optimization problem, but converges slowly in theoretical analysis. To solve this problem, there are already many algorithms with variance reduction techniques, such as SVRG, SAG, SAGA, etc. Some scholars apply the conjugate gradient method in traditional optimization to these algorithms, such as CGVR, SCGA, SCGN, etc., which can basically achieve linear convergence speed, but these conclusions often need to be established under some relatively strong assumptions. In traditional optimization, the conjugate gradient method often requires the use of line search techniques to achieve good experimental results. In a sense, line search embodies some properties of the conjugate methods. Taking inspiration from this, we apply the modified three-term conjugate gradient method and line search technique to machine learning. In our theoretical analysis, we obtain the same convergence rate as SCGA under weaker conditional assumptions. We also test the convergence of our algorithm using two non-convex machine learning models.","PeriodicalId":22058,"journal":{"name":"Statistics and Computing","volume":"27 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140311638","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Novel sampling method for the von Mises–Fisher distribution von Mises-Fisher 分布的新型抽样方法

IF 2.2 2区数学

Statistics and Computing Pub Date : 2024-03-26 DOI: 10.1007/s11222-024-10419-3

引用次数: 0

Generalized spherical principal component analysis 广义球形主成分分析

IF 2.2 2区数学

Statistics and Computing Pub Date : 2024-03-23 DOI: 10.1007/s11222-024-10413-9

Sarah Leyder, Jakob Raymaekers, Tim Verdonck

引用次数: 0

An expectile computation cookbook 预期计算食谱

IF 2.2 2区数学

Statistics and Computing Pub Date : 2024-03-23 DOI: 10.1007/s11222-024-10403-x

引用次数: 0

Variable selection using axis-aligned random projections for partial least-squares regression 利用轴对齐随机投影为部分最小二乘回归选择变量

IF 2.2 2区数学

Statistics and Computing Pub Date : 2024-03-23 DOI: 10.1007/s11222-024-10417-5

{"title":"Variable selection using axis-aligned random projections for partial least-squares regression","authors":"","doi":"10.1007/s11222-024-10417-5","DOIUrl":"https://doi.org/10.1007/s11222-024-10417-5","url":null,"abstract":"<h3>Abstract</h3> In high-dimensional data modeling, variable selection plays a crucial role in improving predictive accuracy and enhancing model interpretability through sparse representation. Unfortunately, certain variable selection methods encounter challenges such as insufficient model sparsity, high computational overhead, and difficulties in handling large-scale data. Recently, axis-aligned random projection techniques have been applied to address these issues by selecting variables. However, these techniques have seen limited application in handling complex data within the regression framework. In this study, we propose a novel method, sparse partial least squares via axis-aligned random projection, designed for the analysis of high-dimensional data. Initially, axis-aligned random projection is utilized to obtain a sparse loading vector, significantly reducing computational complexity. Subsequently, partial least squares regression is conducted within the subspace of the top-ranked significant variables. The submatrices are iteratively updated until an optimal sparse partial least squares model is achieved. Comparative analysis with some state-of-the-art high-dimensional regression methods demonstrates that the proposed method exhibits superior predictive performance. To illustrate its effectiveness, we apply the method to four cases, including one simulated dataset and three real-world datasets. The results show the proposed method’s ability to identify important variables in all four cases.","PeriodicalId":22058,"journal":{"name":"Statistics and Computing","volume":"39 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-03-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140201284","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Simultaneous estimation and variable selection for a non-crossing multiple quantile regression using deep neural networks 利用深度神经网络实现非交叉多元量级回归的同步估计和变量选择

IF 2.2 2区数学

Statistics and Computing Pub Date : 2024-03-22 DOI: 10.1007/s11222-024-10418-4

Jungmin Shin, Seunghyun Gwak, Seung Jun Shin, Sungwan Bang

引用次数: 0

Resampling-based confidence intervals and bands for the average treatment effect in observational studies with competing risks 有竞争风险的观察性研究中平均治疗效果的基于重采样的置信区间和带宽

IF 2.2 2区数学

Statistics and Computing Pub Date : 2024-03-21 DOI: 10.1007/s11222-024-10420-w

Jasmin Rühl, Sarah Friedrich

{"title":"Resampling-based confidence intervals and bands for the average treatment effect in observational studies with competing risks","authors":"Jasmin Rühl, Sarah Friedrich","doi":"10.1007/s11222-024-10420-w","DOIUrl":"https://doi.org/10.1007/s11222-024-10420-w","url":null,"abstract":"The g-formula can be used to estimate the treatment effect while accounting for confounding bias in observational studies. With regard to time-to-event endpoints, possibly subject to competing risks, the construction of valid pointwise confidence intervals and time-simultaneous confidence bands for the causal risk difference is complicated, however. A convenient solution is to approximate the asymptotic distribution of the corresponding stochastic process by means of resampling approaches. In this paper, we consider three different resampling methods, namely the classical nonparametric bootstrap, the influence function equipped with a resampling approach as well as a martingale-based bootstrap version, the so-called wild bootstrap. For the latter, three sub-versions based on differing distributions of the underlying random multipliers are examined. We set up a simulation study to compare the accuracy of the different techniques, which reveals that the wild bootstrap should in general be preferred if the sample size is moderate and sufficient data on the event of interest have been accrued. For illustration, the resampling methods are further applied to data on the long-term survival in patients with early-stage Hodgkin’s disease.","PeriodicalId":22058,"journal":{"name":"Statistics and Computing","volume":"4 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140201223","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A constant-per-iteration likelihood ratio test for online changepoint detection for exponential family models 指数族模型在线变化点检测的每次迭代恒定似然比检验

IF 2.2 2区数学

Statistics and Computing Pub Date : 2024-03-19 DOI: 10.1007/s11222-024-10416-6

Kes Ward, Gaetano Romano, Idris Eckley, Paul Fearnhead

{"title":"A constant-per-iteration likelihood ratio test for online changepoint detection for exponential family models","authors":"Kes Ward, Gaetano Romano, Idris Eckley, Paul Fearnhead","doi":"10.1007/s11222-024-10416-6","DOIUrl":"https://doi.org/10.1007/s11222-024-10416-6","url":null,"abstract":"Online changepoint detection algorithms that are based on (generalised) likelihood-ratio tests have been shown to have excellent statistical properties. However, a simple online implementation is computationally infeasible as, at time T, it involves considering O(T) possible locations for the change. Recently, the FOCuS algorithm has been introduced for detecting changes in mean in Gaussian data that decreases the per-iteration cost to (O(log T)). This is possible by using pruning ideas, which reduce the set of changepoint locations that need to be considered at time T to approximately (log T). We show that if one wishes to perform the likelihood ratio test for a different one-parameter exponential family model, then exactly the same pruning rule can be used, and again one need only consider approximately (log T) locations at iteration T. Furthermore, we show how we can adaptively perform the maximisation step of the algorithm so that we need only maximise the test statistic over a small subset of these possible locations. Empirical results show that the resulting online algorithm, which can detect changes under a wide range of models, has a constant-per-iteration cost on average.","PeriodicalId":22058,"journal":{"name":"Statistics and Computing","volume":"41 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140168815","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Improving model choice in classification: an approach based on clustering of covariance matrices 改进分类中的模型选择：基于协方差矩阵聚类的方法

IF 2.2 2区数学

Statistics and Computing Pub Date : 2024-03-19 DOI: 10.1007/s11222-024-10410-y

David Rodríguez-Vítores, Carlos Matrán

{"title":"Improving model choice in classification: an approach based on clustering of covariance matrices","authors":"David Rodríguez-Vítores, Carlos Matrán","doi":"10.1007/s11222-024-10410-y","DOIUrl":"https://doi.org/10.1007/s11222-024-10410-y","url":null,"abstract":"This work introduces a refinement of the Parsimonious Model for fitting a Gaussian Mixture. The improvement is based on the consideration of clusters of the involved covariance matrices according to a criterion, such as sharing Principal Directions. This and other similarity criteria that arise from the spectral decomposition of a matrix are the bases of the Parsimonious Model. We show that such groupings of covariance matrices can be achieved through simple modifications of the CEM (Classification Expectation Maximization) algorithm. Our approach leads to propose Gaussian Mixture Models for model-based clustering and discriminant analysis, in which covariance matrices are clustered according to a parsimonious criterion, creating intermediate steps between the fourteen widely known parsimonious models. The added versatility not only allows us to obtain models with fewer parameters for fitting the data, but also provides greater interpretability. We show its usefulness for model-based clustering and discriminant analysis, providing algorithms to find approximate solutions verifying suitable size, shape and orientation constraints, and applying them to both simulation and real data examples.","PeriodicalId":22058,"journal":{"name":"Statistics and Computing","volume":"99 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-03-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140169060","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Functional mixtures-of-experts 专家功能混合物

IF 2.2 2区数学

Statistics and Computing Pub Date : 2024-03-18 DOI: 10.1007/s11222-023-10379-0

Faïcel Chamroukhi, Nhat Thien Pham, Van Hà Hoang, Geoffrey J. McLachlan

{"title":"Functional mixtures-of-experts","authors":"Faïcel Chamroukhi, Nhat Thien Pham, Van Hà Hoang, Geoffrey J. McLachlan","doi":"10.1007/s11222-023-10379-0","DOIUrl":"https://doi.org/10.1007/s11222-023-10379-0","url":null,"abstract":"We consider the statistical analysis of heterogeneous data for prediction, in situations where the observations include functions, typically time series. We extend the modeling with mixtures-of-experts (ME), as a framework of choice in modeling heterogeneity in data for prediction with vectorial observations, to this functional data analysis context. We first present a new family of ME models, named functional ME (FME), in which the predictors are potentially noisy observations, from entire functions. Furthermore, the data generating process of the predictor and the real response, is governed by a hidden discrete variable representing an unknown partition. Second, by imposing sparsity on derivatives of the underlying functional parameters via Lasso-like regularizations, we provide sparse and interpretable functional representations of the FME models called iFME. We develop dedicated expectation–maximization algorithms for Lasso-like regularized maximum-likelihood parameter estimation strategies to fit the models. The proposed models and algorithms are studied in simulated scenarios and in applications to two real data sets, and the obtained results demonstrate their performance in accurately capturing complex nonlinear relationships and in clustering the heterogeneous regression data.","PeriodicalId":22058,"journal":{"name":"Statistics and Computing","volume":"114 1","pages":""},"PeriodicalIF":2.2,"publicationDate":"2024-03-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140152764","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0