Journal of Machine Learning Research最新文献_第8页

An l ∞ Eigenvector Perturbation Bound and Its Application to Robust Covariance Estimation. 一个l∞特征向量扰动界及其在鲁棒协方差估计中的应用。

IF 4.3 3区计算机科学

Journal of Machine Learning Research Pub Date : 2018-04-01

Jianqing Fan, Weichen Wang, Yiqiao Zhong

{"title":"<ArticleTitle xmlns:ns0=\"http://www.w3.org/1998/Math/MathML\">An <ns0:math> <ns0:mrow><ns0:msub><ns0:mi>l</ns0:mi> <ns0:mi>∞</ns0:mi></ns0:msub> </ns0:mrow> </ns0:math> Eigenvector Perturbation Bound and Its Application to Robust Covariance Estimation.","authors":"Jianqing Fan, Weichen Wang, Yiqiao Zhong","doi":"","DOIUrl":"","url":null,"abstract":"In statistics and machine learning, we are interested in the eigenvectors (or singular vectors) of certain matrices (e.g. covariance matrices, data matrices, etc). However, those matrices are usually perturbed by noises or statistical errors, either from random sampling or structural patterns. The Davis-Kahan sin θ theorem is often used to bound the difference between the eigenvectors of a matrix A and those of a perturbed matrix <math> <mrow><mover><mi>A</mi> <mo>˜</mo></mover> <mo>=</mo> <mi>A</mi> <mo>+</mo> <mi>E</mi></mrow> </math> , in terms of <math> <mrow><msub><mi>l</mi> <mn>2</mn></msub> </mrow> </math> norm. In this paper, we prove that when A is a low-rank and incoherent matrix, the <math> <mrow><msub><mi>l</mi> <mi>∞</mi></msub> </mrow> </math> norm perturbation bound of singular vectors (or eigenvectors in the symmetric case) is smaller by a factor of <math> <mrow> <msqrt> <mrow><msub><mi>d</mi> <mn>1</mn></msub> </mrow> </msqrt> </mrow> </math> or <math> <mrow> <msqrt> <mrow><msub><mi>d</mi> <mn>2</mn></msub> </mrow> </msqrt> </mrow> </math> for left and right vectors, where d 1 and d 2 are the matrix dimensions. The power of this new perturbation result is shown in robust covariance estimation, particularly when random variables have heavy tails. There, we propose new robust covariance estimators and establish their asymptotic properties using the newly developed perturbation bound. Our theoretical results are verified through extensive numerical experiments.","PeriodicalId":50161,"journal":{"name":"Journal of Machine Learning Research","volume":null,"pages":null},"PeriodicalIF":4.3,"publicationDate":"2018-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6867801/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49684379","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Simultaneous Clustering and Estimation of Heterogeneous Graphical Models. 异构图形模型的同步聚类和估算。

IF 6 3区计算机科学

Journal of Machine Learning Research Pub Date : 2018-04-01

Botao Hao, Will Wei Sun, Yufeng Liu, Guang Cheng

{"title":"Simultaneous Clustering and Estimation of Heterogeneous Graphical Models.","authors":"Botao Hao, Will Wei Sun, Yufeng Liu, Guang Cheng","doi":"","DOIUrl":"","url":null,"abstract":"We consider joint estimation of multiple graphical models arising from heterogeneous and high-dimensional observations. Unlike most previous approaches which assume that the cluster structure is given in advance, an appealing feature of our method is to learn cluster structure while estimating heterogeneous graphical models. This is achieved via a high dimensional version of Expectation Conditional Maximization (ECM) algorithm (Meng and Rubin, 1993). A joint graphical lasso penalty is imposed on the conditional maximization step to extract both homogeneity and heterogeneity components across all clusters. Our algorithm is computationally efficient due to fast sparse learning routines and can be implemented without unsupervised learning knowledge. The superior performance of our method is demonstrated by extensive experiments and its application to a Glioblastoma cancer dataset reveals some new insights in understanding the Glioblastoma cancer. In theory, a non-asymptotic error bound is established for the output directly from our high dimensional ECM algorithm, and it consists of two quantities: statistical error (statistical accuracy) and optimization error (computational complexity). Such a result gives a theoretical guideline in terminating our ECM iterations.","PeriodicalId":50161,"journal":{"name":"Journal of Machine Learning Research","volume":null,"pages":null},"PeriodicalIF":6.0,"publicationDate":"2018-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6338433/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"36923362","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Robust Learning Approach for Regression Models Based on Distributionally Robust Optimization. 基于分布鲁棒优化的回归模型鲁棒学习方法。

IF 6 3区计算机科学

Journal of Machine Learning Research Pub Date : 2018-01-01

Ruidi Chen, Ioannis Ch Paschalidis

{"title":"A Robust Learning Approach for Regression Models Based on Distributionally Robust Optimization.","authors":"Ruidi Chen, Ioannis Ch Paschalidis","doi":"","DOIUrl":"","url":null,"abstract":"We present a Distributionally Robust Optimization (DRO) approach to estimate a robustified regression plane in a linear regression setting, when the observed samples are potentially contaminated with adversarially corrupted outliers. Our approach mitigates the impact of outliers by hedging against a family of probability distributions on the observed data, some of which assign very low probabilities to the outliers. The set of distributions under consideration are close to the empirical distribution in the sense of the Wasserstein metric. We show that this DRO formulation can be relaxed to a convex optimization problem which encompasses a class of models. By selecting proper norm spaces for the Wasserstein metric, we are able to recover several commonly used regularized regression models. We provide new insights into the regularization term and give guidance on the selection of the regularization coefficient from the standpoint of a confidence region. We establish two types of performance guarantees for the solution to our formulation under mild conditions. One is related to its out-of-sample behavior (prediction bias), and the other concerns the discrepancy between the estimated and true regression planes (estimation bias). Extensive numerical results demonstrate the superiority of our approach to a host of regression models, in terms of the prediction and estimation accuracies. We also consider the application of our robust learning procedure to outlier detection, and show that our approach achieves a much higher AUC (Area Under the ROC Curve) than M-estimation (Huber, 1964, 1973).","PeriodicalId":50161,"journal":{"name":"Journal of Machine Learning Research","volume":null,"pages":null},"PeriodicalIF":6.0,"publicationDate":"2018-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8378760/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"39333876","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A constructive approach to L0 penalized regression 一种建设性的L0惩罚回归方法

IF 6 3区计算机科学

Journal of Machine Learning Research Pub Date : 2018-01-01 DOI: 10.5555/3291125.3291135

HuangJian, JiaoYuling, liuyanyan, LuXiliang

引用次数: 45

Invariant models for causal transfer learning 因果迁移学习的不变模型

IF 6 3区计算机科学

Journal of Machine Learning Research Pub Date : 2018-01-01 DOI: 10.5555/3291125.3291161

Rojas-CarullaMateo, SchölkopfBernhard, TurnerRichard, PetersJonas

引用次数: 1

Auto-WEKA 2.0: Automatic model selection and hyperparameter optimization in WEKA Auto-WEKA 2.0:在WEKA中实现自动模型选择和超参数优化

IF 6 3区计算机科学

Journal of Machine Learning Research Pub Date : 2017-01-01 DOI: 10.1007/978-3-030-05318-5_4

Lars Kotthoff, C. Thornton, H. Hoos, F. Hutter, Kevin Leyton-Brown

引用次数: 644

Learning Scalable Deep Kernels with Recurrent Structure. 用循环结构学习可扩展深度核。

IF 6 3区计算机科学

Journal of Machine Learning Research Pub Date : 2017-01-01

Maruan Al-Shedivat, Andrew Gordon Wilson, Yunus Saatchi, Zhiting Hu, Eric P Xing

引用次数: 0

Empirical evaluation of resampling procedures for optimising SVM hyperparameters 优化支持向量机超参数的重采样过程的经验评价

IF 6 3区计算机科学

Journal of Machine Learning Research Pub Date : 2017-01-01 DOI: 10.5555/3122009.3122024

WainerJacques, CawleyGavin

引用次数: 0

Community Extraction in Multilayer Networks with Heterogeneous Community Structure. 具有异构社区结构的多层网络中的社区提取。

IF 6 3区计算机科学

Journal of Machine Learning Research Pub Date : 2017-01-01

James D Wilson, John Palowitch, Shankar Bhamidi, Andrew B Nobel

{"title":"Community Extraction in Multilayer Networks with Heterogeneous Community Structure.","authors":"James D Wilson, John Palowitch, Shankar Bhamidi, Andrew B Nobel","doi":"","DOIUrl":"","url":null,"abstract":"Multilayer networks are a useful way to capture and model multiple, binary or weighted relationships among a fixed group of objects. While community detection has proven to be a useful exploratory technique for the analysis of single-layer networks, the development of community detection methods for multilayer networks is still in its infancy. We propose and investigate a procedure, called Multilayer Extraction, that identifies densely connected vertex-layer sets in multilayer networks. Multilayer Extraction makes use of a significance based score that quantifies the connectivity of an observed vertex-layer set through comparison with a fixed degree random graph model. Multilayer Extraction directly handles networks with heterogeneous layers where community structure may be different from layer to layer. The procedure can capture overlapping communities, as well as background vertex-layer pairs that do not belong to any community. We establish consistency of the vertex-layer set optimizer of our proposed multilayer score under the multilayer stochastic block model. We investigate the performance of Multilayer Extraction on three applications and a test bed of simulations. Our theoretical and numerical evaluations suggest that Multilayer Extraction is an effective exploratory tool for analyzing complex multilayer networks. Publicly available code is available at https://github.com/jdwilson4/MultilayerExtraction.","PeriodicalId":50161,"journal":{"name":"Journal of Machine Learning Research","volume":null,"pages":null},"PeriodicalIF":6.0,"publicationDate":"2017-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6927681/pdf/nihms-1022819.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"37486356","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Automatic differentiation in machine learning 机器学习中的自动微分

IF 6 3区计算机科学

Journal of Machine Learning Research Pub Date : 2017-01-01 DOI: 10.5555/3122009.3242010

BaydinAtılım Günes, A. PearlmutterBarak, RadulAlexey Andreyevich, S. Mark

引用次数: 23