{"title":"Ensuring Rapid Mixing and Low Bias for Asynchronous Gibbs Sampling","authors":"Christopher De Sa, Christopher Ré, K. Olukotun","doi":"10.24963/ijcai.2017/672","DOIUrl":"https://doi.org/10.24963/ijcai.2017/672","url":null,"abstract":"Gibbs sampling is a Markov chain Monte Carlo technique commonly used for estimating marginal distributions. To speed up Gibbs sampling, there has recently been interest in parallelizing it by executing asynchronously. While empirical results suggest that many models can be efficiently sampled asynchronously, traditional Markov chain analysis does not apply to the asynchronous case, and thus asynchronous Gibbs sampling is poorly understood. In this paper, we derive a better understanding of the two main challenges of asynchronous Gibbs: bias and mixing time. We show experimentally that our theoretical results match practical outcomes.","PeriodicalId":89793,"journal":{"name":"JMLR workshop and conference proceedings","volume":"1 1","pages":"1567-1576"},"PeriodicalIF":0.0,"publicationDate":"2016-02-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"77475373","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Ensuring Rapid Mixing and Low Bias for Asynchronous Gibbs Sampling.","authors":"Christopher De Sa, Kunle Olukotun, Christopher Ré","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Gibbs sampling is a Markov chain Monte Carlo technique commonly used for estimating marginal distributions. To speed up Gibbs sampling, there has recently been interest in parallelizing it by executing asynchronously. While empirical results suggest that many models can be efficiently sampled asynchronously, traditional Markov chain analysis does not apply to the asynchronous case, and thus asynchronous Gibbs sampling is poorly understood. In this paper, we derive a better understanding of the two main challenges of asynchronous Gibbs: bias and mixing time. We show experimentally that our theoretical results match practical outcomes.</p>","PeriodicalId":89793,"journal":{"name":"JMLR workshop and conference proceedings","volume":"48 ","pages":"1567-1576"},"PeriodicalIF":0.0,"publicationDate":"2016-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5360990/pdf/nihms826682.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"34857287","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ian E H Yen, Xin Lin, Jiong Zhang, Pradeep Ravikumar, Inderjit S Dhillon
{"title":"A Convex Atomic-Norm Approach to Multiple Sequence Alignment and Motif Discovery.","authors":"Ian E H Yen, Xin Lin, Jiong Zhang, Pradeep Ravikumar, Inderjit S Dhillon","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Multiple Sequence Alignment and Motif Discovery, known as NP-hard problems, are two fundamental tasks in Bioinformatics. Existing approaches to these two problems are based on either local search methods such as Expectation Maximization (EM), Gibbs Sampling or greedy heuristic methods. In this work, we develop a convex relaxation approach to both problems based on the recent concept of atomic norm and develop a new algorithm, termed Greedy Direction Method of Multiplier, for solving the convex relaxation with two convex atomic constraints. Experiments show that our convex relaxation approach produces solutions of higher quality than those standard tools widely-used in Bioinformatics community on the Multiple Sequence Alignment and Motif Discovery problems.</p>","PeriodicalId":89793,"journal":{"name":"JMLR workshop and conference proceedings","volume":"48 ","pages":"2272-2280"},"PeriodicalIF":0.0,"publicationDate":"2016-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4993214/pdf/nihms808905.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"34389307","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Sandhya Prabhakaran, Elham Azizi, Ambrose Carr, Dana Pe'er
{"title":"Dirichlet Process Mixture Model for Correcting Technical Variation in Single-Cell Gene Expression Data.","authors":"Sandhya Prabhakaran, Elham Azizi, Ambrose Carr, Dana Pe'er","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>We introduce an iterative normalization and clustering method for single-cell gene expression data. The emerging technology of single-cell RNA-seq gives access to gene expression measurements for thousands of cells, allowing discovery and characterization of cell types. However, the data is confounded by technical variation emanating from experimental errors and cell type-specific biases. Current approaches perform a global normalization prior to analyzing biological signals, which does not resolve missing data or variation dependent on latent cell types. Our model is formulated as a hierarchical Bayesian mixture model with cell-specific scalings that aid the iterative normalization and clustering of cells, teasing apart technical variation from biological signals. We demonstrate that this approach is superior to global normalization followed by clustering. We show identifiability and weak convergence guarantees of our method and present a scalable Gibbs inference algorithm. This method improves cluster inference in both synthetic and real single-cell data compared with previous methods, and allows easy interpretation and recovery of the underlying structure and cell types.</p>","PeriodicalId":89793,"journal":{"name":"JMLR workshop and conference proceedings","volume":"48 ","pages":"1070-1079"},"PeriodicalIF":0.0,"publicationDate":"2016-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6004614/pdf/nihms972080.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"36243698","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Manifold-valued Dirichlet Processes.","authors":"Hyunwoo J Kim, Jia Xu, Baba C Vemuri, Vikas Singh","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Statistical models for manifold-valued data permit capturing the intrinsic nature of the curved spaces in which the data lie and have been a topic of research for several decades. Typically, these formulations use geodesic curves and distances defined <i>locally</i> for most cases - this makes it hard to design parametric models <i>globally</i> on smooth manifolds. Thus, most (manifold specific) parametric models available today assume that the data lie in a small neighborhood on the manifold. To address this 'locality' problem, we propose a novel nonparametric model which unifies multivariate general linear models (MGLMs) using multiple tangent spaces. Our framework generalizes existing work on (both Euclidean and non-Euclidean) general linear models providing a recipe to globally extend the locally-defined parametric models (using a mixture of local models). By grouping observations into sub-populations at multiple tangent spaces, our method provides insights into the hidden structure (geodesic relationships) in the data. This yields a framework to group observations and discover geodesic relationships between covariates <i>X</i> and manifold-valued responses <i>Y</i>, which we call Dirichlet process mixtures of multivariate general linear models (DP-MGLM) on Riemannian manifolds. Finally, we present proof of concept experiments to validate our model.</p>","PeriodicalId":89793,"journal":{"name":"JMLR workshop and conference proceedings","volume":"2015 ","pages":"1199-1208"},"PeriodicalIF":0.0,"publicationDate":"2015-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4783460/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"72212239","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Huitong Qiu, Sheng Xu, Fang Han, Han Liu, Brian Caffo
{"title":"Robust Estimation of Transition Matrices in High Dimensional Heavy-tailed Vector Autoregressive Processes.","authors":"Huitong Qiu, Sheng Xu, Fang Han, Han Liu, Brian Caffo","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Gaussian vector autoregressive (VAR) processes have been extensively studied in the literature. However, Gaussian assumptions are stringent for heavy-tailed time series that frequently arises in finance and economics. In this paper, we develop a unified framework for modeling and estimating heavy-tailed VAR processes. In particular, we generalize the Gaussian VAR model by an elliptical VAR model that naturally accommodates heavy-tailed time series. Under this model, we develop a quantile-based robust estimator for the transition matrix of the VAR process. We show that the proposed estimator achieves parametric rates of convergence in high dimensions. This is the first work in analyzing heavy-tailed high dimensional VAR processes. As an application of the proposed framework, we investigate Granger causality in the elliptical VAR process, and show that the robust transition matrix estimator induces sign-consistent estimators of Granger causality. The empirical performance of the proposed methodology is demonstrated by both synthetic and real data. We show that the proposed estimator is robust to heavy tails, and exhibit superior performance in stock price prediction.</p>","PeriodicalId":89793,"journal":{"name":"JMLR workshop and conference proceedings","volume":"37 ","pages":"1843-1851"},"PeriodicalIF":0.0,"publicationDate":"2015-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5266499/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"89720992","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jie Liu, Chunming Zhang, Elizabeth Burnside, David Page
{"title":"Multiple Testing under Dependence via Semiparametric Graphical Models.","authors":"Jie Liu, Chunming Zhang, Elizabeth Burnside, David Page","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>It has been shown that graphical models can be used to leverage the dependence in large-scale multiple testing problems with significantly improved performance (Sun & Cai, 2009; Liu et al., 2012). These graphical models are fully parametric and require that we know the parameterization of <i>f</i><sub>1</sub> - the density function of the test statistic under the alternative hypothesis. However in practice, <i>f</i><sub>1</sub> is often heterogeneous, and cannot be estimated with a simple parametric distribution. We propose a novel semiparametric approach for multiple testing under dependence, which estimates <i>f</i><sub>1</sub> adaptively. This semiparametric approach exactly generalizes the local FDR procedure (Efron et al., 2001) and connects with the BH procedure (Benjamini & Hochberg, 1995). A variety of simulations show that our semiparametric approach outperforms classical procedures which assume independence and the parametric approaches which capture dependence.</p>","PeriodicalId":89793,"journal":{"name":"JMLR workshop and conference proceedings","volume":"32 2","pages":"955-963"},"PeriodicalIF":0.0,"publicationDate":"2014-12-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4190841/pdf/nihms612860.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"32741386","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Spherical Hamiltonian Monte Carlo for Constrained Target Distributions.","authors":"Shiwei Lan, Bo Zhou, Babak Shahbaba","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Statistical models with constrained probability distributions are abundant in machine learning. Some examples include regression models with norm constraints (e.g., Lasso), probit models, many copula models, and Latent Dirichlet Allocation (LDA) models. Bayesian inference involving probability distributions confined to constrained domains could be quite challenging for commonly used sampling algorithms. For such problems, we propose a novel Markov Chain Monte Carlo (MCMC) method that provides a general and computationally efficient framework for handling boundary conditions. Our method first maps the <i>D</i>-dimensional constrained domain of parameters to the unit ball [Formula: see text], then augments it to a <i>D</i>-dimensional sphere <b>S</b><sup><i>D</i></sup> such that the original boundary corresponds to the equator of <b>S</b><sup><i>D</i></sup> . This way, our method handles the constraints implicitly by moving freely on the sphere generating proposals that remain within boundaries when mapped back to the original space. To improve the computational efficiency of our algorithm, we divide the dynamics into several parts such that the resulting split dynamics has a partial analytical solution as a geodesic flow on the sphere. We apply our method to several examples including truncated Gaussian, Bayesian Lasso, Bayesian bridge regression, and a copula model for identifying synchrony among multiple neurons. Our results show that the proposed method can provide a natural and efficient framework for handling several types of constraints on target distributions.</p>","PeriodicalId":89793,"journal":{"name":"JMLR workshop and conference proceedings","volume":"32 ","pages":"629-637"},"PeriodicalIF":0.0,"publicationDate":"2014-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4407381/pdf/nihms672830.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"33133055","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hadi Daneshmand, Manuel Gomez-Rodriguez, Le Song, Bernhard Schölkopf
{"title":"Estimating Diffusion Network Structures: Recovery Conditions, Sample Complexity & Soft-thresholding Algorithm.","authors":"Hadi Daneshmand, Manuel Gomez-Rodriguez, Le Song, Bernhard Schölkopf","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Information spreads across social and technological networks, but often the network structures are hidden from us and we only observe the traces left by the diffusion processes, called <i>cascades</i>. Can we recover the hidden network structures from these observed cascades? What kind of cascades and how many cascades do we need? Are there some network structures which are more difficult than others to recover? Can we design efficient inference algorithms with provable guarantees? Despite the increasing availability of cascade-data and methods for inferring networks from these data, a thorough theoretical understanding of the above questions remains largely unexplored in the literature. In this paper, we investigate the network structure inference problem for a general family of continuous-time diffusion models using an [Formula: see text]-regularized likelihood maximization framework. We show that, as long as the cascade sampling process satisfies a natural incoherence condition, our framework can recover the correct network structure with high probability if we observe <i>O</i>(<i>d</i><sup>3</sup> log <i>N</i>) cascades, where <i>d</i> is the maximum number of parents of a node and <i>N</i> is the total number of nodes. Moreover, we develop a simple and efficient soft-thresholding inference algorithm, which we use to illustrate the consequences of our theoretical results, and show that our framework outperforms other alternatives in practice.</p>","PeriodicalId":89793,"journal":{"name":"JMLR workshop and conference proceedings","volume":"32 2","pages":"793-801"},"PeriodicalIF":0.0,"publicationDate":"2014-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4412853/pdf/nihms-680553.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"33147202","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nan Du, Yingyu Liang, Maria-Florina Balcan, Le Song
{"title":"Influence Function Learning in Information Diffusion Networks.","authors":"Nan Du, Yingyu Liang, Maria-Florina Balcan, Le Song","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Can we learn the influence of a set of people in a social network from cascades of information diffusion? This question is often addressed by a two-stage approach: first learn a diffusion model, and then calculate the influence based on the learned model. Thus, the success of this approach relies heavily on the correctness of the diffusion model which is hard to verify for real world data. In this paper, we exploit the insight that the influence functions in many diffusion models are coverage functions, and propose a novel parameterization of such functions using a convex combination of random basis functions. Moreover, we propose an efficient maximum likelihood based algorithm to learn such functions directly from cascade data, and hence bypass the need to specify a particular diffusion model in advance. We provide both theoretical and empirical analysis for our approach, showing that the proposed approach can provably learn the influence function with low sample complexity, be robust to the unknown diffusion models, and significantly outperform existing approaches in both synthetic and real world data.</p>","PeriodicalId":89793,"journal":{"name":"JMLR workshop and conference proceedings","volume":"32 2","pages":"2016-2024"},"PeriodicalIF":0.0,"publicationDate":"2014-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4427574/pdf/nihms680551.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"33303443","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}