IEEE Transactions on Information Theory最新文献

{"title":"Information Divergences and Likelihood Ratios of Poisson Processes and Point Patterns","authors":"Lasse Leskelä","doi":"10.1109/TIT.2024.3472448","DOIUrl":"https://doi.org/10.1109/TIT.2024.3472448","url":null,"abstract":"This article develops an analytical framework for studying information divergences and likelihood ratios associated with Poisson processes and point patterns on general measurable spaces. The main results include explicit analytical formulas for Kullback-Leibler divergences, Rényi divergences, Hellinger distances, and likelihood ratios of the laws of Poisson point patterns in terms of their intensity measures. The general results yield similar formulas for inhomogeneous Poisson processes, compound Poisson processes, as well as spatial and marked Poisson point patterns. Additional results include simple characterisations of absolute continuity, mutual singularity, and the existence of common dominating measures. The analytical toolbox is based on Tsallis divergences of sigma-finite measures on abstract measurable spaces. The treatment is purely information-theoretic and free of topological assumptions.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 12","pages":"9084-9101"},"PeriodicalIF":2.2,"publicationDate":"2024-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10703134","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142713914","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Bias-Corrected Joint Spectral Embedding for Multilayer Networks With Invariant Subspace: Entrywise Eigenvector Perturbation and Inference","authors":"Fangzheng Xie","doi":"10.1109/TIT.2024.3471953","DOIUrl":"https://doi.org/10.1109/TIT.2024.3471953","url":null,"abstract":"In this paper, we propose to estimate the invariant subspace across heterogeneous multiple networks using a novel bias-corrected joint spectral embedding algorithm. The proposed algorithm recursively calibrates the diagonal bias of the sum of squared network adjacency matrices by leveraging the closed-form bias formula and iteratively updates the subspace estimator using the most recent estimated bias. Correspondingly, we establish a complete recipe for the entrywise subspace estimation theory for the proposed algorithm, including a sharp entrywise subspace perturbation bound and the entrywise eigenvector central limit theorem. Leveraging these results, we settle two multiple network inference problems: the exact community detection in multilayer stochastic block models and the hypothesis testing of the equality of membership profiles in multilayer mixed membership models. Our proof relies on delicate leave-one-out and leave-two-out analyses that are specifically tailored to block-wise symmetric random matrices and a martingale argument that is of fundamental interest for the entrywise eigenvector central limit theorem.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 12","pages":"9036-9083"},"PeriodicalIF":2.2,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142713881","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Sparse Vector and Low Rank Recovery Phase Transitions: Uncovering the Explicit Relations","authors":"Agostino Capponi;Mihailo Stojnic","doi":"10.1109/TIT.2024.3471746","DOIUrl":"https://doi.org/10.1109/TIT.2024.3471746","url":null,"abstract":"We investigate the two primary categories of structured recovery problems, namely Compressed Sensing (CS) and Low Rank Recovery (LRR). Our focus is on the performance analysis of their two tightest convex relaxation based heuristics, the so-called \u0000<inline-formula> <tex-math>$ell _{1}$ </tex-math></inline-formula>\u0000 and the nuclear norm (\u0000<inline-formula> <tex-math>$ell _{1}^{*}$ </tex-math></inline-formula>\u0000) minimizations. We examine two standard types of phase transitions (PTs): 1) general PT, obtained by enforcing sparsity as a fundamental form of structuring, and 2) nonnegative PT, achieved by imposing nonnegativity as an additional form of structuring alongside sparsity. We establish explicit relations between the CS and LRR PTs. Our analysis reveals that the nonnegative PT essentially interpolates between the general and the binary CS PT, in a manner that can be explicitly characterized. Quite surprisingly, although the phase transitions themselves admit fairly complicated mathematical formulations, their relations can be expressed in a very neat and elegant way. This ultimately allows to quickly assess and compare the effects additional presence/absence of the nonnegativity has on \u0000<inline-formula> <tex-math>$ell _{1}$ </tex-math></inline-formula>\u0000 and \u0000<inline-formula> <tex-math>$ell _{1}^{*}$ </tex-math></inline-formula>\u0000.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 12","pages":"9239-9260"},"PeriodicalIF":2.2,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142713883","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A New Achievable Region of the K-User MAC Wiretap Channel With Confidential and Open Messages Under Strong Secrecy","authors":"Hao Xu;Kai-Kit Wong;Giuseppe Caire","doi":"10.1109/TIT.2024.3471662","DOIUrl":"https://doi.org/10.1109/TIT.2024.3471662","url":null,"abstract":"This paper investigates the achievable region of a K-user discrete memoryless (DM) multiple access wiretap (MAC-WT) channel, where each user transmits both secret and open (i.e., non-confidential) messages. All these messages are intended for the legitimate receiver (Bob), while the eavesdropper (Eve) is only interested in the secret messages. In the achievable coding strategy, the confidential information is protected by open messages and also by the introduction of auxiliary messages. When introducing an auxiliary message, one has to ensure that, on one hand, its rate is large enough for protecting the secret message from Eve and, on the other hand, the resulting sum rate (together with the secret and open message rate) does not exceed Bob’s decoding capability. This yields an inequality structure involving the rates of all users’ secret, open, and auxiliary messages. To obtain the rate region, the auxiliary message rates must be eliminated from the system of inequalities. A direct application of the Fourier-Motzkin elimination procedure is elusive since a) it requires that the number of users K is explicitly given, and b) even for small \u0000<inline-formula> <tex-math>$K = 3, 4, ldots $ </tex-math></inline-formula>\u0000, the number of inequalities becomes extremely large. We prove the result for general K through the combined use of Fourier-Motzkin elimination procedure and mathematical induction. This paper adopts the strong secrecy metric, characterized by information leakage. To prove the achievability under this criterion, we analyze the resolvability region of a K-user DM-MAC channel (not necessarily a wiretap channel). In addition, we show that users with zero secrecy rate can play different roles and use different strategies in encoding their messages. These strategies yield non-redundant (i.e., not mutually dominating) rate inequalities. By considering all possible coding strategies, we provide a new achievable region for the considered channel, and show that it strictly improves those already known in the existing literature by considering a specific example.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 12","pages":"9123-9151"},"PeriodicalIF":2.2,"publicationDate":"2024-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142713762","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Covariance-Based Activity Detection in Cooperative Multi-Cell Massive MIMO: Scaling Law and Efficient Algorithms","authors":"Ziyue Wang;Ya-Feng Liu;Zhaorui Wang;Wei Yu","doi":"10.1109/TIT.2024.3470952","DOIUrl":"https://doi.org/10.1109/TIT.2024.3470952","url":null,"abstract":"This paper focuses on the covariance-based activity detection problem in a multi-cell massive multiple-input multiple-output (MIMO) system. In this system, active devices transmit their signature sequences to multiple base stations (BSs), and the BSs cooperatively detect the active devices based on the received signals. While the scaling law for the covariance-based activity detection in the single-cell scenario has been extensively analyzed in the literature, this paper aims to analyze the scaling law for the covariance-based activity detection in the multi-cell massive MIMO system. Specifically, this paper demonstrates a quadratic scaling law in the multi-cell system, under the assumption that the path-loss exponent of the fading channel \u0000<inline-formula> <tex-math>$gamma gt 2$ </tex-math></inline-formula>\u0000. This finding shows that, in the multi-cell massive MIMO system, the maximum number of active devices that can be correctly detected in each cell increases quadratically with the length of the signature sequence and decreases logarithmically with the number of cells (as the number of antennas tends to infinity). Moreover, in addition to analyzing the scaling law for the signature sequences randomly and uniformly distributed on a sphere, the paper also establishes the scaling law for signature sequences based on a finite alphabet, which are easier to generate and store. Finally, this paper proposes two efficient accelerated coordinate descent (CD) algorithms with a convergence guarantee for solving the device activity detection problem. The first algorithm reduces the complexity of CD by using an inexact coordinate update strategy. The second algorithm avoids unnecessary computations of CD by using an active set selection strategy. Simulation results show that the proposed algorithms exhibit excellent performance in terms of computational efficiency and detection error probability.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 12","pages":"8770-8790"},"PeriodicalIF":2.2,"publicationDate":"2024-09-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142713841","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Derivatives of Entropy and the MMSE Conjecture","authors":"Paul Mansanarez;Guillaume Poly;Yvik Swan","doi":"10.1109/TIT.2024.3466566","DOIUrl":"https://doi.org/10.1109/TIT.2024.3466566","url":null,"abstract":"We investigate the properties of the entropy of a probability measure along the heat flow and more precisely we seek for closed algebraic representations of its derivatives. Provided that the measure admits moments of any order, it has been proved in Guo et al. (2010) that this functional is smooth, and in Ledoux (2016) that its derivatives at zero can be expressed into multivariate polynomials evaluated in the moments (or cumulants) of the underlying measure. Moreover, these algebraic expressions are derived through \u0000<inline-formula> <tex-math>$Gamma $ </tex-math></inline-formula>\u0000-calculus techniques which provide implicit recursive formulas for these polynomials. Our main contribution consists in a fine combinatorial analysis of these inductive relations and for the first time to derive closed formulas for the leading coefficients of these polynomials expressions. Building upon these explicit formulas we revisit the so-called “MMSE conjecture” from Ledoux (2016) which asserts that two distributions on the real line with the same entropy along the heat flow must coincide up to translation and symmetry. Our approach enables us to provide new conditions on the source distributions ensuring that the MMSE conjecture holds and to refine several criteria proved in Ledoux (2016). As illustrating examples, our findings cover the cases of uniform and Rademacher distributions, for which previous results in the literature were inapplicable.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 11","pages":"7647-7663"},"PeriodicalIF":2.2,"publicationDate":"2024-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142518123","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Rate-Distortion-Perception Tradeoff Based on the Conditional-Distribution Perception Measure","authors":"Sadaf Salehkalaibar;Jun Chen;Ashish Khisti;Wei Yu","doi":"10.1109/TIT.2024.3467282","DOIUrl":"https://doi.org/10.1109/TIT.2024.3467282","url":null,"abstract":"This paper studies the rate-distortion-perception (RDP) tradeoff for a memoryless source model in the asymptotic limit of large block-lengths. The perception measure is based on a divergence between the distributions of the source and reconstruction sequences conditioned on the encoder output, first proposed by Mentzer et al. We consider the case when there is no shared randomness between the encoder and the decoder and derive a single-letter characterization of the RDP function, for the case of discrete memoryless sources. This is in contrast to the marginal-distribution metric case (introduced by Blau and Michaeli), whose RDP characterization remains open when there is no shared randomness. The achievability scheme is based on lossy source coding with a posterior reference map. For the case of continuous valued sources under the squared error distortion measure and the squared quadratic Wasserstein perception measure, we also derive a single-letter characterization and show that the decoder can be restricted to a noise-adding mechanism. Interestingly, the RDP function characterized for the case of zero perception loss coincides with that of the marginal metric, and further zero perception loss can be achieved with a 3-dB penalty in minimum distortion. Finally we specialize to the case of Gaussian sources, and derive the RDP function for Gaussian vector case and propose a reverse water-filling type solution. We also partially characterize the RDP function for a mixture of Gaussian vector sources.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 12","pages":"8432-8454"},"PeriodicalIF":2.2,"publicationDate":"2024-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142753871","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Large and Small Deviations for Statistical Sequence Matching","authors":"Lin Zhou;Qianyun Wang;Jingjing Wang;Lin Bai;Alfred O. Hero","doi":"10.1109/TIT.2024.3464586","DOIUrl":"https://doi.org/10.1109/TIT.2024.3464586","url":null,"abstract":"We revisit the problem of statistical sequence matching between two databases of sequences initiated by Unnikrishnan, (2015) and derive theoretical performance guarantees for the generalized likelihood ratio test (GLRT). We first consider the case where the number of matched pairs of sequences between the databases is known. In this case, the task is to accurately find the matched pairs of sequences among all possible matches between the sequences in the two databases. We analyze the performance of the GLRT by Unnikrishnan and explicitly characterize the tradeoff between the mismatch and false reject probabilities under each hypothesis in both large and small deviations regimes. Furthermore, we demonstrate the optimality of Unnikrishnan’s GLRT test under the generalized Neyman-Person criterion for both regimes and illustrate our theoretical results via numerical examples. Subsequently, we generalize our achievability analyses to the case where the number of matched pairs is unknown, and an additional error probability needs to be considered. When one of the two databases contains a single sequence, the problem of statistical sequence matching specializes to the problem of multiple classification introduced by Gutman, (1989). For this special case, our result for the small deviations regime strengthens previous result of Zhou et al., (2020) by removing unnecessary conditions on the generating distributions.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 11","pages":"7532-7562"},"PeriodicalIF":2.2,"publicationDate":"2024-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142517965","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Kernel Stein Discrepancy on Lie Groups: Theory and Applications","authors":"Xiaoda Qu;Xiran Fan;Baba C. Vemuri","doi":"10.1109/TIT.2024.3468212","DOIUrl":"https://doi.org/10.1109/TIT.2024.3468212","url":null,"abstract":"Distributional approximation is a fundamental problem in machine learning with numerous applications across all fields of science and engineering and beyond. The key challenge in most approximation methods is the need to tackle the intractable normalization constant present in the candidate distributions used to model the data. This intractability is especially common in distributions of manifold-valued random variables such as rotation matrices, orthogonal matrices etc. In this paper, we focus on the distributional approximation problem in Lie groups since they are frequently encountered in many applications including but not limited to, computer vision, robotics, medical imaging and many more. We present a novel Stein’s operator on Lie groups leading to a kernel Stein discrepancy (KSD), which is a normalization-free loss function. We present several theoretical results characterizing the properties of this new KSD on Lie groups and its minimizer namely, the minimum KSD estimator (MKSDE). Properties of MKSDE are presented and proved, including strong consistency, CLT and a closed form of the MKSDE for the von Mises-Fisher and in general, the exponential family on \u0000<inline-formula> <tex-math>$mathop {mathrm {SO}}nolimits (N)$ </tex-math></inline-formula>\u0000. Finally, we present several experimental results depicting advantages of MKSDE over maximum likelihood estimation.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 12","pages":"8961-8974"},"PeriodicalIF":2.2,"publicationDate":"2024-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142713756","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Mean and Variance of the Reciprocal Merit Factor of Four Classes of Binary Sequences","authors":"Jonathan Jedwab","doi":"10.1109/TIT.2024.3467194","DOIUrl":"https://doi.org/10.1109/TIT.2024.3467194","url":null,"abstract":"The merit factor of a \u0000<inline-formula> <tex-math>${-1, 1}$ </tex-math></inline-formula>\u0000 binary sequence measures the collective smallness of its non-trivial aperiodic autocorrelations. Binary sequences with large merit factor are important in digital communications because they allow the efficient separation of signals from noise. It is a longstanding open question whether the maximum merit factor is asymptotically unbounded and, if so, what is its limiting value. Attempts to answer this question over almost sixty years have identified certain classes of binary sequences as particularly important: skew-symmetric sequences, symmetric sequences, and anti-symmetric sequences. Using only elementary methods, we find an exact formula for the mean and variance of the reciprocal merit factor of sequences in each of these classes, and in the class of all binary sequences. This provides a much deeper understanding of the distribution of the merit factor in these four classes than was previously available. A consequence is that, for each of the four classes, the merit factor of a sequence drawn uniformly at random from the class converges in probability to a constant as the sequence length increases.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"70 12","pages":"9227-9238"},"PeriodicalIF":2.2,"publicationDate":"2024-09-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142713842","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}