IEEE Transactions on Information Theory最新文献

{"title":"Intersection and Union of Subspaces With Applications to Communication Over Authenticated Classical-Quantum Channels and Composite Hypothesis Testing","authors":"Naqueeb Ahmad Warsi;Ayanava Dasgupta","doi":"10.1109/TIT.2025.3531565","DOIUrl":"https://doi.org/10.1109/TIT.2025.3531565","url":null,"abstract":"In information theory, we often use intersection and union of the typical sets to analyze various communication problems. However, in the quantum setting it is not very clear how to construct a measurement which behaves analogously to intersection and union of the typical sets. In this work, we construct a projection operator which behaves very similarly to intersection and union of the typical sets. Our construction relies on the Jordan’s lemma. Using this construction we study the problem of communication over authenticated classical-quantum channels and derive its capacity. As another application of our construction, we also study the problem of quantum asymmetric composite hypothesis testing.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 4","pages":"2763-2781"},"PeriodicalIF":2.2,"publicationDate":"2025-01-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143667309","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Asymptotically Optimal Codes for (t, s)-Burst Error","authors":"Yubo Sun;Ziyang Lu;Yiwei Zhang;Gennian Ge","doi":"10.1109/TIT.2025.3531915","DOIUrl":"https://doi.org/10.1109/TIT.2025.3531915","url":null,"abstract":"Recently, codes for correcting a burst of errors have attracted significant attention. One of the most important reasons is that bursts of errors occur in certain emerging techniques, such as DNA storage. In this paper, we investigate a type of error, called a <inline-formula> <tex-math>$(t,s)$ </tex-math></inline-formula>-burst, which deletes t consecutive symbols and inserts s arbitrary symbols at the same coordinate. Note that a <inline-formula> <tex-math>$(t,s)$ </tex-math></inline-formula>-burst error can be seen as a generalization of a burst of insertions (<inline-formula> <tex-math>$t=0$ </tex-math></inline-formula>), a burst of deletions (<inline-formula> <tex-math>$s=0$ </tex-math></inline-formula>), and a burst of substitutions (<inline-formula> <tex-math>$t=s$ </tex-math></inline-formula>). Our main contribution is to give explicit constructions of q-ary <inline-formula> <tex-math>$(t,s)$ </tex-math></inline-formula>-burst correcting codes with <inline-formula> <tex-math>$log n + O(1)$ </tex-math></inline-formula> bits of redundancy for any given constant non-negative integers t, s, and <inline-formula> <tex-math>$q geq 2$ </tex-math></inline-formula>. These codes have optimal redundancy up to an additive constant. Furthermore, we apply our <inline-formula> <tex-math>$(t,s)$ </tex-math></inline-formula>-burst correcting codes to combat other various types of errors and improve the corresponding results. In particular, one of our byproducts is a permutation code capable of correcting a burst of t stable deletions with <inline-formula> <tex-math>$log n + O(1)$ </tex-math></inline-formula> bits of redundancy, which is optimal up to an additive constant.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 3","pages":"1570-1584"},"PeriodicalIF":2.2,"publicationDate":"2025-01-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143465644","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Limiting Moments of Autocorrelation Demerit Factors of Binary Sequences","authors":"Daniel J. Katz;Miriam E. Ramirez","doi":"10.1109/TIT.2025.3532557","DOIUrl":"https://doi.org/10.1109/TIT.2025.3532557","url":null,"abstract":"Various problems in engineering and natural science demand binary sequences that do not resemble translates of themselves, that is, the sequences must have small aperiodic autocorrelation at every nonzero shift. If f is a sequence, then the demerit factor of f is the sum of the squared magnitudes of the autocorrelations at all nonzero shifts for the sequence obtained by normalizing f to unit Euclidean norm. The demerit factor is the reciprocal of Golay’s merit factor, and low demerit factor indicates low self-similarity of a sequence under translation. We endow the <inline-formula> <tex-math>$2^{ell } $ </tex-math></inline-formula> binary sequences of length <inline-formula> <tex-math>$ell $ </tex-math></inline-formula> with uniform probability measure and consider the distribution of their demerit factors. Earlier works used combinatorial techniques to find exact formulas for the mean, variance, skewness, and kurtosis of the distribution as a function of <inline-formula> <tex-math>$ell $ </tex-math></inline-formula>. These revealed that for <inline-formula> <tex-math>$ell geq 4$ </tex-math></inline-formula>, the pth central moment of this distribution is strictly positive for every <inline-formula> <tex-math>$p geq 2$ </tex-math></inline-formula>. This article shows that for every p, the pth central moment is <inline-formula> <tex-math>$ell ^{-2 p}$ </tex-math></inline-formula> times a quasi-polynomial function of <inline-formula> <tex-math>$ell $ </tex-math></inline-formula> with rational coefficients. It also shows that, in the limit as <inline-formula> <tex-math>$ell $ </tex-math></inline-formula> tends to infinity, the pth standardized moment is the same as that of the standard normal distribution.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 4","pages":"3181-3193"},"PeriodicalIF":2.2,"publicationDate":"2025-01-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143667313","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Generalization Performance of Empirical Risk Minimization on Over-Parameterized Deep ReLU Nets","authors":"Shao-Bo Lin;Yao Wang;Ding-Xuan Zhou","doi":"10.1109/TIT.2025.3531048","DOIUrl":"https://doi.org/10.1109/TIT.2025.3531048","url":null,"abstract":"In this paper, we study the generalization performance of global minima of empirical risk minimization (ERM) on over-parameterized deep ReLU nets. Using a novel deepening scheme for deep ReLU nets, we rigorously prove that there exist perfect global minima achieving optimal generalization error rates for numerous types of data under mild conditions. Since over-parameterization of deep ReLU nets is crucial to guarantee that the global minima of ERM can be realized by the widely used stochastic gradient descent (SGD) algorithm, our results present a potential way to fill the gap between optimization and generalization of deep learning.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 3","pages":"1978-1993"},"PeriodicalIF":2.2,"publicationDate":"2025-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10844907","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143455274","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Semantic Security With Infinite-Dimensional Quantum Eavesdropping Channel","authors":"Matthias Frey;Igor Bjelaković;Janis Nötzel;Sławomir Stańczak","doi":"10.1109/TIT.2025.3531250","DOIUrl":"https://doi.org/10.1109/TIT.2025.3531250","url":null,"abstract":"We propose a new proof method for direct coding theorems for wiretap channels where the eavesdropper has access to a quantum version of the transmitted signal on an infinite-dimensional Hilbert space and the legitimate parties communicate through a classical channel or a classical input, quantum output (cq) channel. The transmitter input can be subject to an additive cost constraint, which specializes to the case of an average energy constraint. This method yields errors that decay exponentially with increasing block lengths. Moreover, it provides a guarantee of a quantum version of semantic security, which is an established concept in classical cryptography and physical layer security. Therefore, it complements existing works which either do not prove the exponential error decay or use weaker notions of security. The main part of this proof method is a direct coding result on channel resolvability which states that there is only a doubly exponentially small probability that a standard random codebook does not solve the channel resolvability problem for the cq channel. Semantic security has strong operational implications meaning essentially that the eavesdropper cannot use its quantum observation to gather any meaningful information about the transmitted signal. We also discuss the connections between semantic security and various other established notions of secrecy.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 4","pages":"2662-2697"},"PeriodicalIF":2.2,"publicationDate":"2025-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143676091","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Proving Information Inequalities by Gaussian Elimination","authors":"Laigang Guo;Raymond W. Yeung;Xiao-Shan Gao","doi":"10.1109/TIT.2025.3531133","DOIUrl":"https://doi.org/10.1109/TIT.2025.3531133","url":null,"abstract":"The proof of information inequalities and identities under linear constraints on the information measures is an important problem in information theory. For this purpose, ITIP and other variant algorithms have been developed and implemented, which are all based on solving a linear program (LP). Building on our recent work (Guo et al., 2023), we developed in this paper an enhanced approach for solving this problem. Experimental results show that our new approach improves the time complexity by over 500 times compared with Guo et al. (2023) for the problem studied by Tian (2014).","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 4","pages":"2315-2328"},"PeriodicalIF":2.2,"publicationDate":"2025-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143676039","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"How Does Distribution Matching Help Domain Generalization: An Information-Theoretic Analysis","authors":"Yuxin Dong;Tieliang Gong;Hong Chen;Shuangyong Song;Weizhan Zhang;Chen Li","doi":"10.1109/TIT.2025.3531136","DOIUrl":"https://doi.org/10.1109/TIT.2025.3531136","url":null,"abstract":"Domain generalization aims to learn invariance across multiple source domains, thereby enhancing generalization against out-of-distribution data. While gradient or representation matching algorithms have achieved remarkable success in domain generalization, these methods generally lack generalization guarantees or depend on strong assumptions, leaving a gap in understanding the underlying mechanism of distribution matching. In this work, we formulate domain generalization from a novel probabilistic perspective, ensuring robustness while avoiding overly conservative solutions. Through comprehensive information-theoretic analysis, we provide key insights into the roles of gradient and representation matching in promoting generalization. Our results reveal the complementary relationship between these two components, indicating that existing works focusing solely on either gradient or representation alignment are insufficient to solve the domain generalization problem. In light of these theoretical findings, we introduce IDM to simultaneously align the inter-domain gradients and representations. Integrated with the proposed PDM method for complex distribution matching, IDM achieves superior performance over various baseline methods.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 3","pages":"2028-2053"},"PeriodicalIF":2.2,"publicationDate":"2025-01-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143455273","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Codes for Correcting a Burst of Edits Using Weighted-Summation VT Sketch","authors":"Yubo Sun;Gennian Ge","doi":"10.1109/TIT.2025.3530506","DOIUrl":"https://doi.org/10.1109/TIT.2025.3530506","url":null,"abstract":"Bursts of errors are a class of errors that can be found in a variety of applications. A burst of t edits refers to a burst of t deletions, or a burst of t insertions, or a burst of t substitutions. This paper focuses on studying codes that can correct a burst of t edits. Our primary approach involves the use of the tool called weighted-summation VT sketch. The <inline-formula> <tex-math>$(t,k)$ </tex-math></inline-formula>-weighted-summation VT sketch of a length-n sequence is defined as the weighted summation of the VT sketch of each row of its <inline-formula> <tex-math>$ttimes lceil n/t rceil $ </tex-math></inline-formula> array representation, with weights in the i-th row set as <inline-formula> <tex-math>$k^{i-1}$ </tex-math></inline-formula> for <inline-formula> <tex-math>$i=1,2,ldots,t$ </tex-math></inline-formula>. By employing the weighted-summation VT sketch alongside multiple weight sketches, we introduce a construction for q-ary t-burst-substitution correcting codes with a redundancy of <inline-formula> <tex-math>$log n+O(1)$ </tex-math></inline-formula>, where the logarithm base is 2. Subsequently, we improve the redundancy to address specific types of burst-substitution errors, such as inversion errors, adjacent-block-transposition errors, and absorption errors. Moreover, by utilizing the method developed in the construction of burst-substitution correcting codes and imposing additional run-length-limited constraints, locally-bounded constraints, and strong-locally-balanced constraints, respectively, we introduce three constructions of t-burst-deletion correcting codes, each requiring a redundancy of <inline-formula> <tex-math>$log n+O(log log n)$ </tex-math></inline-formula>. Any t-burst-deletion-correcting code is also a t-burst-insertion correcting code, allowing us to intersect the t-burst-substitution-correcting codes and t-burst-deletion-correcting codes designed above to derive three constructions of q-ary t-burst-edit-correcting codes. The first two constructions have a redundancy of <inline-formula> <tex-math>$log n+(tlog q-1)log log n+O(1)$ </tex-math></inline-formula>, while the third construction has a redundancy of <inline-formula> <tex-math>$log n+log log n+O(1)$ </tex-math></inline-formula>. Most of the proposed codes demonstrate superior performance compared to previous results, with the exception of burst-deletion correcting codes. Furthermore, in cases of single-edit errors (t-burst-edit error with <inline-formula> <tex-math>$t=1$ </tex-math></inline-formula>), the redundancy of the first two constructions of quaternary single-edit correcting codes outperforms the results of Gabrys et al. (IEEE Trans. Inf. Theory 2023). We also provide efficient encoding and decoding algorithms for our codes to enhance their practical usability.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 3","pages":"1631-1646"},"PeriodicalIF":2.2,"publicationDate":"2025-01-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143465668","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Private Noisy Side Information Helps to Increase the Capacity of SPIR","authors":"Hassan ZivariFard;Rémi A. Chou;Xiaodong Wang","doi":"10.1109/TIT.2025.3530400","DOIUrl":"https://doi.org/10.1109/TIT.2025.3530400","url":null,"abstract":"Noiseless private side information does not reduce the download cost in Symmetric Private Information Retrieval (SPIR) unless the client knows all but one file. While this is a pessimistic result, we explore in this paper whether noisy client side information available at the client helps decrease the download cost in the context of SPIR with colluding and replicated servers. Specifically, we assume that the client possesses noisy side information about each stored file, which is obtained by passing each file through one of D possible discrete memoryless test channels. The statistics of the test channels are known by the client and by all the servers, but the mapping <inline-formula> <tex-math>$boldsymbol {mathcal {M}}$ </tex-math></inline-formula> between the files and the test channels is unknown to the servers. We study this problem under two privacy metrics. Under the first metric, the client wants to preserve the privacy of its file selection and the mapping <inline-formula> <tex-math>$boldsymbol {mathcal {M}}$ </tex-math></inline-formula>, and the servers want to preserve the privacy of all the non-selected files. Under the second metric, the client is willing to reveal the index of the test channel that is associated with its desired file. For both privacy metrics, we derive the optimal common randomness and download cost. Our setup generalizes SPIR with colluding servers and SPIR with private noiseless side information. Unlike noiseless side information, our results demonstrate that noisy side information can reduce the download cost, even when the client does not have noiseless knowledge of all but one file.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 3","pages":"2140-2156"},"PeriodicalIF":2.2,"publicationDate":"2025-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143455254","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Implicit Balancing and Regularization: Generalization and Convergence Guarantees for Overparameterized Asymmetric Matrix Sensing","authors":"Mahdi Soltanolkotabi;Dominik Stöger;Changzhi Xie","doi":"10.1109/TIT.2025.3530335","DOIUrl":"https://doi.org/10.1109/TIT.2025.3530335","url":null,"abstract":"Recently, there has been significant progress in understanding the convergence and generalization properties of gradient-based methods for training overparameterized learning models. However, many aspects including the role of small random initialization and how the various parameters of the model are coupled during gradient-based updates to facilitate good generalization remain largely mysterious. A series of recent papers have begun to study this role for non-convex formulations of symmetric Positive Semi-Definite (PSD) matrix sensing problems which involve reconstructing a low-rank PSD matrix from a few linear measurements. The underlying symmetry/PSDness is crucial to existing convergence and generalization guarantees for this problem. In this paper, we study a general overparameterized low-rank matrix sensing problem where one wishes to reconstruct an asymmetric rectangular low-rank matrix from a few linear measurements. We prove that an overparameterized model trained via factorized gradient descent converges to the low-rank matrix generating the measurements. We show that in this setting, factorized gradient descent enjoys two implicit properties: (1) coupling of the trajectory of gradient descent where the factors are coupled in various ways throughout the gradient update trajectory and (2) an algorithmic regularization property where the iterates show a propensity towards low-rank models despite the overparameterized nature of the factorized model. These two implicit properties in turn allow us to show that the gradient descent trajectory from small random initialization moves towards solutions that are both globally optimal and generalize well.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 4","pages":"2991-3037"},"PeriodicalIF":2.2,"publicationDate":"2025-01-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143667499","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}