Annals of Statistics最新文献_第4页

SURPRISES IN HIGH-DIMENSIONAL RIDGELESS LEAST SQUARES INTERPOLATION. 高维无脊最小二乘插值中的奇异值。

IF 4.5 1区数学

Annals of Statistics Pub Date : 2022-04-01 Epub Date: 2022-04-07 DOI: 10.1214/21-aos2133

Trevor Hastie, Andrea Montanari, Saharon Rosset, Ryan J Tibshirani

{"title":"SURPRISES IN HIGH-DIMENSIONAL RIDGELESS LEAST SQUARES INTERPOLATION.","authors":"Trevor Hastie, Andrea Montanari, Saharon Rosset, Ryan J Tibshirani","doi":"10.1214/21-aos2133","DOIUrl":"https://doi.org/10.1214/21-aos2133","url":null,"abstract":"Interpolators-estimators that achieve zero training error-have attracted growing attention in machine learning, mainly because state-of-the art neural networks appear to be models of this type. In this paper, we study minimum ℓ 2 norm (\"ridgeless\") interpolation least squares regression, focusing on the high-dimensional regime in which the number of unknown parameters p is of the same order as the number of samples n. We consider two different models for the feature distribution: a linear model, where the feature vectors <math> <mrow><msub><mi>x</mi> <mi>i</mi></msub> <mo>∈</mo> <msup><mi>ℝ</mi> <mi>p</mi></msup> </mrow> </math> are obtained by applying a linear transform to a vector of i.i.d. entries, x i = Σ1/2 z i (with <math> <mrow><msub><mi>z</mi> <mi>i</mi></msub> <mo>∈</mo> <msup><mi>ℝ</mi> <mi>p</mi></msup> </mrow> </math> ); and a nonlinear model, where the feature vectors are obtained by passing the input through a random one-layer neural network, xi = φ(Wz i ) (with <math> <mrow><msub><mi>z</mi> <mi>i</mi></msub> <mo>∈</mo> <msup><mi>ℝ</mi> <mi>d</mi></msup> </mrow> </math> , <math><mrow><mi>W</mi> <mo>∈</mo> <msup><mi>ℝ</mi> <mrow><mi>p</mi> <mo>×</mo> <mi>d</mi></mrow> </msup> </mrow> </math> a matrix of i.i.d. entries, and φ an activation function acting componentwise on Wz i ). We recover-in a precise quantitative way-several phenomena that have been observed in large-scale neural networks and kernel machines, including the \"double descent\" behavior of the prediction risk, and the potential benefits of overparametrization.","PeriodicalId":8032,"journal":{"name":"Annals of Statistics","volume":" ","pages":"949-986"},"PeriodicalIF":4.5,"publicationDate":"2022-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9481183/pdf/nihms-1830540.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"40367700","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 579

OPTIMAL FALSE DISCOVERY RATE CONTROL FOR LARGE SCALE MULTIPLE TESTING WITH AUXILIARY INFORMATION. 基于辅助信息的大规模多重测试的最优错误发现率控制。

IF 4.5 1区数学

Annals of Statistics Pub Date : 2022-04-01 DOI: 10.1214/21-aos2128

Hongyuan Cao, Jun Chen, Xianyang Zhang

{"title":"OPTIMAL FALSE DISCOVERY RATE CONTROL FOR LARGE SCALE MULTIPLE TESTING WITH AUXILIARY INFORMATION.","authors":"Hongyuan Cao, Jun Chen, Xianyang Zhang","doi":"10.1214/21-aos2128","DOIUrl":"https://doi.org/10.1214/21-aos2128","url":null,"abstract":"Large-scale multiple testing is a fundamental problem in high dimensional statistical inference. It is increasingly common that various types of auxiliary information, reflecting the structural relationship among the hypotheses, are available. Exploiting such auxiliary information can boost statistical power. To this end, we propose a framework based on a two-group mixture model with varying probabilities of being null for different hypotheses a priori, where a shape-constrained relationship is imposed between the auxiliary information and the prior probabilities of being null. An optimal rejection rule is designed to maximize the expected number of true positives when average false discovery rate is controlled. Focusing on the ordered structure, we develop a robust EM algorithm to estimate the prior probabilities of being null and the distribution of p-values under the alternative hypothesis simultaneously. We show that the proposed method has better power than state-of-the-art competitors while controlling the false discovery rate, both empirically and theoretically. Extensive simulations demonstrate the advantage of the proposed method. Datasets from genome-wide association studies are used to illustrate the new methodology.","PeriodicalId":8032,"journal":{"name":"Annals of Statistics","volume":"50 2","pages":"807-857"},"PeriodicalIF":4.5,"publicationDate":"2022-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10153594/pdf/nihms-1840915.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9776938","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 15

FUNCTIONAL SUFFICIENT DIMENSION REDUCTION THROUGH AVERAGE FRÉCHET DERIVATIVES. 通过平均弗雷谢特导数实现函数充分降维。

IF 4.5 1区数学

Annals of Statistics Pub Date : 2022-04-01 Epub Date: 2022-04-07 DOI: 10.1214/21-aos2131

Kuang-Yao Lee, Lexin Li

{"title":"FUNCTIONAL SUFFICIENT DIMENSION REDUCTION THROUGH AVERAGE FRÉCHET DERIVATIVES.","authors":"Kuang-Yao Lee, Lexin Li","doi":"10.1214/21-aos2131","DOIUrl":"10.1214/21-aos2131","url":null,"abstract":"Sufficient dimension reduction (SDR) embodies a family of methods that aim for reduction of dimensionality without loss of information in a regression setting. In this article, we propose a new method for nonparametric function-on-function SDR, where both the response and the predictor are a function. We first develop the notions of functional central mean subspace and functional central subspace, which form the population targets of our functional SDR. We then introduce an average Fréchet derivative estimator, which extends the gradient of the regression function to the operator level and enables us to develop estimators for our functional dimension reduction spaces. We show the resulting functional SDR estimators are unbiased and exhaustive, and more importantly, without imposing any distributional assumptions such as the linearity or the constant variance conditions that are commonly imposed by all existing functional SDR methods. We establish the uniform convergence of the estimators for the functional dimension reduction spaces, while allowing both the number of Karhunen-Loève expansions and the intrinsic dimension to diverge with the sample size. We demonstrate the efficacy of the proposed methods through both simulations and two real data examples.","PeriodicalId":8032,"journal":{"name":"Annals of Statistics","volume":"50 2","pages":"904-929"},"PeriodicalIF":4.5,"publicationDate":"2022-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10085580/pdf/nihms-1746366.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"9320340","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Testability of high-dimensional linear models with nonsparse structures. 非稀疏结构高维线性模型的可测试性。

IF 4.5 1区数学

Annals of Statistics Pub Date : 2022-04-01 Epub Date: 2022-04-07 DOI: 10.1214/19-aos1932

Jelena Bradic, Jianqing Fan, Yinchu Zhu

{"title":"Testability of high-dimensional linear models with nonsparse structures.","authors":"Jelena Bradic, Jianqing Fan, Yinchu Zhu","doi":"10.1214/19-aos1932","DOIUrl":"https://doi.org/10.1214/19-aos1932","url":null,"abstract":"Understanding statistical inference under possibly non-sparse high-dimensional models has gained much interest recently. For a given component of the regression coefficient, we show that the difficulty of the problem depends on the sparsity of the corresponding row of the precision matrix of the covariates, not the sparsity of the regression coefficients. We develop new concepts of uniform and essentially uniform non-testability that allow the study of limitations of tests across a broad set of alternatives. Uniform non-testability identifies a collection of alternatives such that the power of any test, against any alternative in the group, is asymptotically at most equal to the nominal size. Implications of the new constructions include new minimax testability results that, in sharp contrast to the current results, do not depend on the sparsity of the regression parameters. We identify new tradeoffs between testability and feature correlation. In particular, we show that, in models with weak feature correlations, minimax lower bound can be attained by a test whose power has the <math> <mrow><msqrt><mi>n</mi></msqrt> </mrow> </math> rate, regardless of the size of the model sparsity.","PeriodicalId":8032,"journal":{"name":"Annals of Statistics","volume":" ","pages":"615-639"},"PeriodicalIF":4.5,"publicationDate":"2022-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9266975/pdf/nihms-1639563.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"40580296","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 11

SEMIPARAMETRIC LATENT-CLASS MODELS FOR MULTIVARIATE LONGITUDINAL AND SURVIVAL DATA. 多变量纵向和生存数据的半参数潜伏类模型。

IF 4.5 1区数学

Annals of Statistics Pub Date : 2022-02-01 Epub Date: 2022-02-16 DOI: 10.1214/21-aos2117

Kin Yau Wong, Donglin Zeng, D Y Lin

引用次数: 0

Spatial dependence and space–time trend in extreme events 极端事件的空间依赖性和时空趋势

IF 4.5 1区数学

Annals of Statistics Pub Date : 2022-02-01 DOI: 10.1214/21-aos2067

John H. J. Einmahl,Ana Ferreira,Laurens de Haan,Cláudia Neves,Chen Zhou

引用次数: 0

CANONICAL THRESHOLDING FOR NON-SPARSE HIGH-DIMENSIONAL LINEAR REGRESSION. 非稀疏高维线性回归的典型阈值。

IF 4.5 1区数学

Annals of Statistics Pub Date : 2022-02-01 Epub Date: 2022-02-16 DOI: 10.1214/21-aos2116

Igor Silin, Jianqing Fan

{"title":"CANONICAL THRESHOLDING FOR NON-SPARSE HIGH-DIMENSIONAL LINEAR REGRESSION.","authors":"Igor Silin, Jianqing Fan","doi":"10.1214/21-aos2116","DOIUrl":"https://doi.org/10.1214/21-aos2116","url":null,"abstract":"We consider a high-dimensional linear regression problem. Unlike many papers on the topic, we do not require sparsity of the regression coefficients; instead, our main structural assumption is a decay of eigenvalues of the covariance matrix of the data. We propose a new family of estimators, called the canonical thresholding estimators, which pick largest regression coefficients in the canonical form. The estimators admit an explicit form and can be linked to LASSO and Principal Component Regression (PCR). A theoretical analysis for both fixed design and random design settings is provided. Obtained bounds on the mean squared error and the prediction error of a specific estimator from the family allow to clearly state sufficient conditions on the decay of eigenvalues to ensure convergence. In addition, we promote the use of the relative errors, strongly linked with the out-of-sample R 2. The study of these relative errors leads to a new concept of joint effective dimension, which incorporates the covariance of the data and the regression coefficients simultaneously, and describes the complexity of a linear regression problem. Some minimax lower bounds are established to showcase the optimality of our procedure. Numerical simulations confirm good performance of the proposed estimators compared to the previously developed methods.","PeriodicalId":8032,"journal":{"name":"Annals of Statistics","volume":" ","pages":"460-486"},"PeriodicalIF":4.5,"publicationDate":"2022-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9491498/pdf/nihms-1782574.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"33478241","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Total variation regularized Fréchet regression for metric-space valued data 度量空间值数据的总变分正则化fr<s:1>回归

IF 4.5 1区数学

Annals of Statistics Pub Date : 2021-12-01 DOI: 10.1214/21-aos2095

Zhenhua Lin,Hans-Georg Müller

{"title":"Total variation regularized Fréchet regression for metric-space valued data","authors":"Zhenhua Lin,Hans-Georg Müller","doi":"10.1214/21-aos2095","DOIUrl":"https://doi.org/10.1214/21-aos2095","url":null,"abstract":"Non-Euclidean data that are indexed with a scalar predictor such as time are increasingly encountered in data applications, while statistical methodology and theory for such random objects are not well developed yet. To address the need for new methodology in this area, we develop a total variation regularization technique for nonparametric Frechet regression, which refers to a regression setting where a response residing in a generic metric space is paired with a scalar predictor and the target is a conditional Frechet mean. Specifically, we seek to approximate an unknown metric-space valued function by an estimator that minimizes the Frechet version of least squares and at the same time has small total variation, appropriately defined for metric-space valued objects. We show that the resulting estimator is representable by a piece-wise constant function and establish the minimax convergence rate of the proposed estimator for metric data objects that reside in Hadamard spaces. We illustrate the numerical performance of the proposed method for both simulated and real data, including metric spaces of symmetric positive-definite matrices with the affine-invariant distance, of probability distributions on the real line with the Wasserstein distance, and of phylogenetic trees with the Billera--Holmes--Vogtmann metric.","PeriodicalId":8032,"journal":{"name":"Annals of Statistics","volume":"74 2","pages":""},"PeriodicalIF":4.5,"publicationDate":"2021-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138508224","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"数学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 13

BRIDGING CONVEX AND NONCONVEX OPTIMIZATION IN ROBUST PCA: NOISE, OUTLIERS, AND MISSING DATA. 在鲁棒 PCA 中连接凸优化和非凸优化：噪声、异常值和缺失数据。

IF 3.2 1区数学

Annals of Statistics Pub Date : 2021-10-01 Epub Date: 2021-11-12 DOI: 10.1214/21-aos2066

Yuxin Chen, Jianqing Fan, Cong Ma, Yuling Yan

引用次数: 0

BOOSTED NONPARAMETRIC HAZARDS WITH TIME-DEPENDENT COVARIATES. 具有时间相关协变量的增强非参数风险。

IF 4.5 1区数学

Annals of Statistics Pub Date : 2021-08-01 Epub Date: 2021-09-29 DOI: 10.1214/20-aos2028

Donald K K Lee, Ningyuan Chen, Hemant Ishwaran

引用次数: 0