Commun. Inf. Syst.最新文献_第2页

Sequential Universal Modeling for Non-Binary Sequences with Constrained Distributions 具有约束分布的非二值序列的序列通用建模

Commun. Inf. Syst. Pub Date : 2020-05-20 DOI: 10.4310/cis.2022.v22.n1.a1

M. Drmota, G. Shamir, W. Szpankowski

{"title":"Sequential Universal Modeling for Non-Binary Sequences with Constrained Distributions","authors":"M. Drmota, G. Shamir, W. Szpankowski","doi":"10.4310/cis.2022.v22.n1.a1","DOIUrl":"https://doi.org/10.4310/cis.2022.v22.n1.a1","url":null,"abstract":"Sequential probability assignment and universal compression go hand in hand. We propose sequential probability assignment for non-binary (and large alphabet) sequences with empirical distributions whose parameters are known to be bounded within a limited interval. Sequential probability assignment algorithms are essential in many applications that require fast and accurate estimation of the maximizing sequence probability. These applications include learning, regression, channel estimation and decoding, prediction, and universal compression. On the other hand, constrained distributions introduce interesting theoretical twists that must be overcome in order to present efficient sequential algorithms. Here, we focus on universal compression for memoryless sources, and present precise analysis for the maximal minimax and the average minimax for constrained distributions. We show that our sequential algorithm based on modified Krichevsky-Trofimov (KT) estimator is asymptotically optimal up to $O(1)$ for both maximal and average redundancies. This paper follows and addresses the challenge presented in cite{stw08} that suggested \"results for the binary case lay the foundation to studying larger alphabets\".","PeriodicalId":185710,"journal":{"name":"Commun. Inf. Syst.","volume":"37 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-05-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"114752958","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Second-Order Guarantees in Centralized, Federated and Decentralized Nonconvex Optimization 集中、联邦和分散非凸优化中的二阶保证

Commun. Inf. Syst. Pub Date : 2020-03-31 DOI: 10.4310/cis.2020.v20.n3.a4

Stefan Vlaski, A. H. Sayed

{"title":"Second-Order Guarantees in Centralized, Federated and Decentralized Nonconvex Optimization","authors":"Stefan Vlaski, A. H. Sayed","doi":"10.4310/cis.2020.v20.n3.a4","DOIUrl":"https://doi.org/10.4310/cis.2020.v20.n3.a4","url":null,"abstract":"Rapid advances in data collection and processing capabilities have allowed for the use of increasingly complex models that give rise to nonconvex optimization problems. These formulations, however, can be arbitrarily difficult to solve in general, in the sense that even simply verifying that a given point is a local minimum can be NP-hard [1]. Still, some relatively simple algorithms have been shown to lead to surprisingly good empirical results in many contexts of interest. Perhaps the most prominent example is the success of the backpropagation algorithm for training neural networks. Several recent works have pursued rigorous analytical justification for this phenomenon by studying the structure of the nonconvex optimization problems and establishing that simple algorithms, such as gradient descent and its variations, perform well in converging towards local minima and avoiding saddle-points. A key insight in these analyses is that gradient perturbations play a critical role in allowing local descent algorithms to efficiently distinguish desirable from undesirable stationary points and escape from the latter. In this article, we cover recent results on second-order guarantees for stochastic first-order optimization algorithms in centralized, federated, and decentralized architectures.","PeriodicalId":185710,"journal":{"name":"Commun. Inf. Syst.","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117351172","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 4

Order Effects of Measurements in Multi-Agent Hypothesis Testing 多智能体假设检验中测量的顺序效应

Commun. Inf. Syst. Pub Date : 2020-03-26 DOI: 10.4310/CIS.2021.V21.N2.A1

Aneesh Raghavan, J. Baras

{"title":"Order Effects of Measurements in Multi-Agent Hypothesis Testing","authors":"Aneesh Raghavan, J. Baras","doi":"10.4310/CIS.2021.V21.N2.A1","DOIUrl":"https://doi.org/10.4310/CIS.2021.V21.N2.A1","url":null,"abstract":"All propositions from the set of events for an agent in a multi-agent system might not be simultaneously verifiable. In this paper, we revisit the concepts of textit{event-state-operation structure} and textit{relationship of incompatibility} from literature and use them as a tool to study the algebraic structure of the set of events. We present an example from multi-agent hypothesis testing where the set of events does not form a Boolean algebra but forms an ortholattice. A possible construction of a 'noncommutative probability space', accounting for textit{incompatible events} (events which cannot be simultaneously verified) is discussed. As a possible decision-making problem in such a probability space, we consider the binary hypothesis testing problem. We present two approaches to this decision-making problem. In the first approach, we represent the available data as coming from measurements modeled via projection valued measures (PVM) and retrieve the results of the underlying detection problem solved using classical probability models. In the second approach, we represent the measurements using positive operator valued measures (POVM). We prove that the minimum probability of error achieved in the second approach is the same as in the first approach.","PeriodicalId":185710,"journal":{"name":"Commun. Inf. Syst.","volume":"47 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2020-03-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127911278","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A global Hartman-Grobman theorem 一个全局Hartman-Grobman定理

Commun. Inf. Syst. Pub Date : 2020-02-14 DOI: 10.4310/cis.2022.v22.n1.a2

Xiaochang A. Wang, Jiexin Dai

引用次数: 2

Zero-sum differential games on the Wasserstein space Wasserstein空间上的零和微分博弈

Commun. Inf. Syst. Pub Date : 2019-12-12 DOI: 10.4310/CIS.2021.V21.N2.A3

T. Başar, Jun Moon

{"title":"Zero-sum differential games on the Wasserstein space","authors":"T. Başar, Jun Moon","doi":"10.4310/CIS.2021.V21.N2.A3","DOIUrl":"https://doi.org/10.4310/CIS.2021.V21.N2.A3","url":null,"abstract":"We consider two-player zero-sum differential games (ZSDGs), where the state process (dynamical system) depends on the random initial condition and the state process's distribution, and the objective functional includes the state process's distribution and the random target variable. Unlike ZSDGs studied in the existing literature, the ZSDG of this paper introduces a new technical challenge, since the corresponding (lower and upper) value functions are defined on $mathcal{P}_2$ (the set of probability measures with finite second moments) or $mathcal{L}_2$ (the set of random variables with finite second moments), both of which are infinite-dimensional spaces. We show that the (lower and upper) value functions on $mathcal{P}_2$ and $mathcal{L}_2$ are equivalent (law invariant) and continuous, satisfying dynamic programming principles. We use the notion of derivative of a function of probability measures in $mathcal{P}_2$ and its lifted version in $mathcal{L}_2$ to show that the (lower and upper) value functions are unique viscosity solutions to the associated (lower and upper) Hamilton-Jacobi-Isaacs equations that are (infinite-dimensional) first-order PDEs on $mathcal{P}_2$ and $mathcal{L}_2$, where the uniqueness is obtained via the comparison principle. Under the Isaacs condition, we show that the ZSDG has a value.","PeriodicalId":185710,"journal":{"name":"Commun. Inf. Syst.","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2019-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129166081","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 3

Whole genome single nucleotide polymorphism genotyping of Staphylococcus aureus 金黄色葡萄球菌全基因组单核苷酸多态性基因分型研究

Commun. Inf. Syst. Pub Date : 2018-10-30 DOI: 10.4310/CIS.2019.V19.N1.A4

Changchuan Yin, S. Yau

{"title":"Whole genome single nucleotide polymorphism genotyping of Staphylococcus aureus","authors":"Changchuan Yin, S. Yau","doi":"10.4310/CIS.2019.V19.N1.A4","DOIUrl":"https://doi.org/10.4310/CIS.2019.V19.N1.A4","url":null,"abstract":"Next-generation sequencing technology enables routine detection of bacterial pathogens for clinical diagnostics and genetic research. Whole genome sequencing has been of importance in the epidemiologic analysis of bacterial pathogens. However, few whole genome sequencing-based genotyping pipelines are available for practical applications. Here, we present the whole genome sequencing-based single nucleotide polymorphism (SNP) genotyping method and apply to the evolutionary analysis of methicillin-resistant Staphylococcus aureus. The SNP genotyping method calls genome variants using next-generation sequencing reads of whole genomes and calculates the pair-wise Jaccard distances of the genome variants. The method may reveal the high-resolution whole genome SNP profiles and the structural variants of different isolates of methicillin-resistant S. aureus (MRSA) and methicillin-susceptible S. aureus (MSSA) strains. The phylogenetic analysis of whole genomes and particular regions may monitor and track the evolution and the transmission dynamic of bacterial pathogens. The computer programs of the whole genome sequencing-based SNP genotyping method are available to the public at this https URL.","PeriodicalId":185710,"journal":{"name":"Commun. Inf. Syst.","volume":"36 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124991497","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 2

Seismic imaging and optimal transport 地震成像和最佳运输

Commun. Inf. Syst. Pub Date : 2018-08-14 DOI: 10.4310/cis.2019.v19.n2.a1

Bjorn Engquist, Yunan Yang

{"title":"Seismic imaging and optimal transport","authors":"Bjorn Engquist, Yunan Yang","doi":"10.4310/cis.2019.v19.n2.a1","DOIUrl":"https://doi.org/10.4310/cis.2019.v19.n2.a1","url":null,"abstract":"Seismology has been an active science for a long time. It changed character about 50 years ago when the earth's vibrations could be measured on the surface more accurately and more frequently in space and time. The full wave field could be determined, and partial differential equations (PDE) started to be used in the inverse process of finding properties of the interior of the earth. We will briefly review earlier techniques but mainly focus on Full Waveform Inversion (FWI) for the acoustic formulation. FWI is a PDE constrained optimization in which the variable velocity in a forward wave equation is adjusted such that the solution matches measured data on the surface. The minimization of the mismatch is usually coupled with the adjoint state method, which also includes the solution to an adjoint wave equation. The least-squares norm is the conventional objective function measuring the difference between simulated and measured data, but it often results in the minimization trapped in local minima. One way to mitigate this is by selecting another misfit function with better convexity properties. Here we propose using the quadratic Wasserstein metric as a new misfit function in FWI. The optimal map defining the quadratic Wasserstein metric can be computed by solving a Monge-Ampere equation. Theorems pointing to the advantages of using optimal transport over the least-squares norm will be discussed, and a number of large-scale computational examples will be presented.","PeriodicalId":185710,"journal":{"name":"Commun. Inf. Syst.","volume":"11 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-08-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131030308","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 14

Understanding Self-Paced Learning under Concave Conjugacy Theory 凹共轭理论下自定节奏学习的理解

Commun. Inf. Syst. Pub Date : 2018-05-01 DOI: 10.4310/CIS.2018.V18.N1.A1

Shiqi Liu, Zilu Ma, Deyu Meng

{"title":"Understanding Self-Paced Learning under Concave Conjugacy Theory","authors":"Shiqi Liu, Zilu Ma, Deyu Meng","doi":"10.4310/CIS.2018.V18.N1.A1","DOIUrl":"https://doi.org/10.4310/CIS.2018.V18.N1.A1","url":null,"abstract":"By simulating the easy-to-hard learning manners of humans/animals, the learning regimes called curriculum learning~(CL) and self-paced learning~(SPL) have been recently investigated and invoked broad interests. However, the intrinsic mechanism for analyzing why such learning regimes can work has not been comprehensively investigated. To this issue, this paper proposes a concave conjugacy theory for looking into the insight of CL/SPL. Specifically, by using this theory, we prove the equivalence of the SPL regime and a latent concave objective, which is closely related to the known non-convex regularized penalty widely used in statistics and machine learning. Beyond the previous theory for explaining CL/SPL insights, this new theoretical framework on one hand facilitates two direct approaches for designing new SPL models for certain tasks, and on the other hand can help conduct the latent objective of self-paced curriculum learning, which is the advanced version of both CL/SPL and possess advantages of both learning regimes to a certain extent. This further facilitates a theoretical understanding for SPCL, instead of only CL/SPL as conventional. Under this theory, we attempt to attain intrinsic latent objectives of two curriculum forms, the partial order and group curriculums, which easily follow the theoretical understanding of the corresponding SPCL regimes.","PeriodicalId":185710,"journal":{"name":"Commun. Inf. Syst.","volume":"24 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123352111","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 8

Rate-distortion functions of non-stationary Markoff chains and their block-independent approximations 非平稳Markoff链的速率畸变函数及其与块无关的近似

Commun. Inf. Syst. Pub Date : 2018-03-27 DOI: 10.4310/cis.2019.v19.n2.a2

Mukul Agarwal

引用次数: 0

Automatic thread painting generation 自动线画生成

Commun. Inf. Syst. Pub Date : 2018-02-13 DOI: 10.4310/CIS.2016.v16.n4.a3

Xiaonan Fang, Bin Liu, Ariel Shamir

引用次数: 0