{"title":"Memory Complexity of Estimating Entropy and Mutual Information","authors":"Tomer Berg;Or Ordentlich;Ofer Shayevitz","doi":"10.1109/TIT.2025.3547871","DOIUrl":"https://doi.org/10.1109/TIT.2025.3547871","url":null,"abstract":"We observe an infinite sequence of independent identically distributed random variables <inline-formula> <tex-math>$X_{1},X_{2},ldots $ </tex-math></inline-formula> drawn from an unknown distribution <italic>p</i> over <inline-formula> <tex-math>$[n]$ </tex-math></inline-formula>, and our goal is to estimate the entropy <inline-formula> <tex-math>$H(p)=-mathop {mathrm {mathbb {E}}}nolimits [log p(X)]$ </tex-math></inline-formula> within an <inline-formula> <tex-math>$varepsilon $ </tex-math></inline-formula>-additive error. To that end, at each time point we are allowed to update a finite-state machine with <italic>S</i> states, using a possibly randomized but time-invariant rule, where each state of the machine is assigned an entropy estimate. Our goal is to characterize the minimax memory complexity <inline-formula> <tex-math>$S^{*}$ </tex-math></inline-formula> of this problem, which is the minimal number of states for which the estimation task is feasible with probability at least <inline-formula> <tex-math>$1-delta $ </tex-math></inline-formula> asymptotically, uniformly in <italic>p</i>. Specifically, we show that there exist universal constants <inline-formula> <tex-math>$C_{1}$ </tex-math></inline-formula> and <inline-formula> <tex-math>$C_{2}$ </tex-math></inline-formula> such that <inline-formula> <tex-math>$ S^{*} leq C_{1}cdot frac {n (log n)^{4}}{varepsilon ^{2}delta }$ </tex-math></inline-formula> for <inline-formula> <tex-math>$varepsilon $ </tex-math></inline-formula> not too small, and <inline-formula> <tex-math>$S^{*} geq C_{2} cdot max left {{{n, frac {log n}{varepsilon }}}right }$ </tex-math></inline-formula> for <inline-formula> <tex-math>$varepsilon $ </tex-math></inline-formula> not too large. The upper bound is proved using approximate counting to estimate the logarithm of <italic>p</i>, and a finite memory bias estimation machine to estimate the expectation operation. The lower bound is proved via a reduction of entropy estimation to uniformity testing. We also apply these results to derive bounds on the memory complexity of mutual information estimation.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 5","pages":"3334-3349"},"PeriodicalIF":2.2,"publicationDate":"2025-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143875188","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Algebraic Characterization of ℳ-Subspaces of Bent Concatenations and Its Application","authors":"Sadmir Kudin;Enes Pasalic;Alexandr Polujan;Fengrong Zhang","doi":"10.1109/TIT.2025.3547533","DOIUrl":"https://doi.org/10.1109/TIT.2025.3547533","url":null,"abstract":"Every Boolean bent function <italic>f</i> can be written either as a concatenation <inline-formula> <tex-math>$f=f_{1}|| f_{2}$ </tex-math></inline-formula> of two complementary semi-bent functions <inline-formula> <tex-math>$f_{1},f_{2}$ </tex-math></inline-formula>; or as a concatenation <inline-formula> <tex-math>$f=f_{1}|| f_{2}|| f_{3}|| f_{4}$ </tex-math></inline-formula> of four Boolean functions <inline-formula> <tex-math>$f_{1},f_{2},f_{3},f_{4}$ </tex-math></inline-formula>, all of which are simultaneously bent, semi-bent, or 5-valued spectra-functions. In this context, it is essential to specify conditions for these bent concatenations so that <italic>f</i> does (not) belong to the completed Maiorana-McFarland class <inline-formula> <tex-math>${mathcal {M}}^{#}$ </tex-math></inline-formula>. In this article, we resolve this question completely by providing the algebraic characterization of <inline-formula> <tex-math>$mathcal {M}$ </tex-math></inline-formula>-subspaces for the concatenation of the form <inline-formula> <tex-math>$f=f_{1}|| f_{2}$ </tex-math></inline-formula> and <inline-formula> <tex-math>$f=f_{1}|| f_{2}|| f_{3}|| f_{4}$ </tex-math></inline-formula>, which allows us to estimate <inline-formula> <tex-math>${rm {ind}}(f)$ </tex-math></inline-formula>, the linearity index of <italic>f</i>, and consequently to establish the necessary and sufficient conditions so that <italic>f</i> is outside <inline-formula> <tex-math>${mathcal {M}}^{#}$ </tex-math></inline-formula>. Based on these conditions, we propose several explicit and generic design methods of specifying bent functions outside <inline-formula> <tex-math>${mathcal {M}}^{#}$ </tex-math></inline-formula> in the special case when <inline-formula> <tex-math>$f=g||h||g||(h+1)$ </tex-math></inline-formula>, where <italic>g</i> and <italic>h</i> are bent functions. Moreover, we show that it is possible to even decrease the linearity index of <inline-formula> <tex-math>$f = g||h||g||(h+1)$ </tex-math></inline-formula>, compared to <inline-formula> <tex-math>${rm {ind}}(g)$ </tex-math></inline-formula> and <inline-formula> <tex-math>${rm {ind}}(h)$ </tex-math></inline-formula>, if the largest dimension of a common <inline-formula> <tex-math>$mathcal {M}$ </tex-math></inline-formula>-subspace of <italic>g</i> and <italic>h</i> is small enough (less than <inline-formula> <tex-math>$min {{rm {ind}}(g), {rm {ind}}(h)} - 1$ </tex-math></inline-formula>). This also induces iterative methods of constructing bent functions outside <inline-formula> <tex-math>${mathcal {M}}^{#}$ </tex-math></inline-formula> with (controllable) low linearity index. Finally, we derive a lower bound on the 2-rank of <italic>f</i> and show that this concatenation method can generate bent functions that are provably outside <inline-formula> <tex-math>${mathcal {M}}^{#} cup {mathcal {PS}}_{ap}^{#}$ </tex-math></inline-formula>. In difference to the approach of Weng et al. (2007) that uses the direct sum and ","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 5","pages":"3999-4011"},"PeriodicalIF":2.2,"publicationDate":"2025-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10909699","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143870979","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"On the Capacity of DNA Labeling","authors":"Dganit Hanania;Daniella Bar-Lev;Yevgeni Nogin;Yoav Shechtman;Eitan Yaakobi","doi":"10.1109/TIT.2025.3545662","DOIUrl":"https://doi.org/10.1109/TIT.2025.3545662","url":null,"abstract":"<italic>DNA labeling</i> is a powerful tool in molecular biology and biotechnology that allows for the visualization, detection, and study of DNA at the molecular level. Under this paradigm, a DNA molecule is being <italic>labeled</i> by specific <italic>k</i> patterns and is then imaged. Then, the resulting image is modeled as a <inline-formula> <tex-math>$(k+1)$ </tex-math></inline-formula>-ary sequence in which any non-zero symbol indicates on the appearance of the corresponding label in the DNA molecule. The primary goal of this work is to study the <italic>labeling capacity</i>, which is defined as the maximal information rate that can be obtained using this labeling process. The labeling capacity is computed for almost any pattern of a single label and several results for multiple labels are provided as well. Moreover, we provide the optimal minimal number of labels of length one or two, over any alphabet of size <italic>q</i>, that are needed in order to achieve the maximum labeling capacity of <inline-formula> <tex-math>$log _{2}(q)$ </tex-math></inline-formula>. Lastly, we discuss the maximal labeling capacity that can be achieved using a certain number of labels of length two.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 5","pages":"3457-3472"},"PeriodicalIF":2.2,"publicationDate":"2025-03-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143875078","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Sharp Signal Detection Under Ferromagnetic Ising Models","authors":"Sohom Bhattacharya;Rajarshi Mukherjee;Gourab Ray","doi":"10.1109/TIT.2025.3546976","DOIUrl":"https://doi.org/10.1109/TIT.2025.3546976","url":null,"abstract":"In this paper, we study a structured signal detection problem in Ferromagnetic Ising models with examples encompassing Ising Models on lattices, and Mean-Field type Ising Models such as dense Erdős-Rényi, and dense random regular graphs. We provide sharp constants of detection in each of these cases and thereby pinpoint an asymptotically precise relationship between the detection problem with the underlying dependence. To obtain this sharp characterization of the detection boundary at the level of sharp multiplicative constants, we derive necessary moderate deviation bounds for partial summands of magnetizations which might be of independent interest. Finally, we demonstrate how our tests can be designed to be adaptive over the strength of dependence present in the respective models.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 5","pages":"3925-3949"},"PeriodicalIF":2.2,"publicationDate":"2025-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143870928","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Computation of the Optimal Error Exponent Function for Fixed-Length Lossy Source Coding in Discrete Memoryless Sources","authors":"Yutaka Jitsumatsu","doi":"10.1109/TIT.2025.3547033","DOIUrl":"https://doi.org/10.1109/TIT.2025.3547033","url":null,"abstract":"Marton’s optimal error exponent for the lossy source coding problem is defined as a non-convex optimization problem. This fact had prevented us to develop an efficient algorithm to compute it. This problem is caused by the fact that the rate-distortion function <inline-formula> <tex-math>$R(Delta |P)$ </tex-math></inline-formula> is potentially non-concave in the probability distribution P for a fixed distortion level <inline-formula> <tex-math>$Delta $ </tex-math></inline-formula>. The main contribution of this paper is the development of a parametric expression that is in perfect agreement with the inverse function of the Marton exponent. This representation has two layers. The inner layer is convex optimization and can be computed efficiently. The outer layer, on the other hand, is a non-convex optimization with respect to two parameters. We give a method for computing the Marton exponent based on this representation.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 5","pages":"3360-3372"},"PeriodicalIF":2.2,"publicationDate":"2025-03-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10908925","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143875184","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Cylindrical and Möbius Quantum Codes for Asymmetric Pauli Errors","authors":"Lorenzo Valentini;Diego Forlivesi;Marco Chiani","doi":"10.1109/TIT.2025.3546769","DOIUrl":"https://doi.org/10.1109/TIT.2025.3546769","url":null,"abstract":"In the implementation of quantum information systems, one type of Pauli error, such as phase-flip errors, may occur more frequently than others, like bit-flip errors. For this reason, quantum error-correcting codes that handle asymmetric errors are critical to mitigating the impact of such impairments. To this aim, several asymmetric quantum codes have been proposed. These include variants of surface codes like the XZZX and ZZZY surface codes, tailored to preserve quantum information in the presence of error asymmetries. In this work, we propose two classes of Calderbank, Shor and Steane (CSS) topological codes, referred to as cylindrical and Möbius codes, particular cases of the fiber bundle family. Cylindrical codes maintain a fully planar structure, while Möbius codes are quasi-planar, with minimal non-local qubit interactions. We construct these codes employing the algebraic chain complexes formalism, providing theoretical upper bounds for the logical error rate. Our results demonstrate that cylindrical and Möbius codes outperform standard surface codes when using the minimum weight perfect matching (MWPM) decoder.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 5","pages":"3766-3778"},"PeriodicalIF":2.2,"publicationDate":"2025-02-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143870887","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"On the Asymptotic Rate of Optimal Codes That Correct Tandem Duplications for Nanopore Sequencing","authors":"Wenjun Yu;Zuo Ye;Moshe Schwartz","doi":"10.1109/TIT.2025.3544875","DOIUrl":"https://doi.org/10.1109/TIT.2025.3544875","url":null,"abstract":"We study codes that can correct backtracking errors during nanopore sequencing. In this channel, a sequence of length <italic>n</i> over an alphabet of size <italic>q</i> is being read by a sliding window of length <inline-formula> <tex-math>$ell $ </tex-math></inline-formula>, where from each window we obtain only its composition. Backtracking errors cause some windows to repeat, hence manifesting as tandem-duplication errors of fixed length <italic>k</i> in the <inline-formula> <tex-math>$ell $ </tex-math></inline-formula>-read vector of window compositions. While existing constructions for duplication-correcting codes can be straightforwardly adapted to this model, even resulting in optimal codes, their asymptotic rate is hard to find. In the regime of unbounded number of duplication errors, we either give the exact asymptotic rate of optimal codes, or bounds on it, depending on the values of <italic>k</i>, <inline-formula> <tex-math>$ell $ </tex-math></inline-formula> and <italic>q</i>. In the regime of a constant number of duplication errors, <italic>t</i>, we find the redundancy of optimal codes to be <inline-formula> <tex-math>$tlog _{q} n+O(1)$ </tex-math></inline-formula> when <inline-formula> <tex-math>$ell |k$ </tex-math></inline-formula>, and only upper bounded by this quantity otherwise.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 5","pages":"3569-3581"},"PeriodicalIF":2.2,"publicationDate":"2025-02-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143875076","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Entanglement-Assisted Covert Communication via Qubit Depolarizing Channels","authors":"Elyakim Zlotnick;Boulat A. Bash;Uzi Pereg","doi":"10.1109/TIT.2025.3546191","DOIUrl":"https://doi.org/10.1109/TIT.2025.3546191","url":null,"abstract":"We consider entanglement-assisted communication over the qubit depolarizing channel under the security requirement of covert communication, where the transmission itself must be concealed from detection by an adversary. Previous work showed that <inline-formula> <tex-math>$O(sqrt {n})$ </tex-math></inline-formula> information bits can be reliably and covertly transmitted in <italic>n</i> channel uses without entanglement assistance. However, Gagatsos et al. (2020) showed that entanglement assistance can increase this scaling to <inline-formula> <tex-math>$O(sqrt {n}log {n})$ </tex-math></inline-formula> for continuous-variable bosonic channels. Here, we present a finite-dimensional parallel, and show that <inline-formula> <tex-math>$O(sqrt {n}log {n})$ </tex-math></inline-formula> covert bits can be transmitted reliably over <italic>n</i> uses of a qubit depolarizing channel. The coding scheme employs “weakly” entangled states such that their squared amplitude scales as <inline-formula> <tex-math>$Oleft ({{{scriptstyle text {}^{scriptstyle 1}}hspace {-0.224em}/hspace {-0.112em}{scriptstyle sqrt {n}}}}right)$ </tex-math></inline-formula>.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 5","pages":"3693-3706"},"PeriodicalIF":2.2,"publicationDate":"2025-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10906331","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143871119","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Novel and Optimal Spectral Method for Permutation Synchronization","authors":"Duc Nguyen;Anderson Ye Zhang","doi":"10.1109/TIT.2025.3545377","DOIUrl":"https://doi.org/10.1109/TIT.2025.3545377","url":null,"abstract":"Permutation synchronization is an important problem in computer science that constitutes the key step of many computer vision tasks. The goal is to recover <italic>n</i> latent permutations from their noisy and incomplete pairwise measurements. In recent years, spectral methods have gained increasing popularity thanks to their simplicity and computational efficiency. Spectral methods utilize the leading eigenspace <italic>U</i> of the data matrix and its block submatrices <inline-formula> <tex-math>$U_{1},U_{2},ldots , U_{n}$ </tex-math></inline-formula> to recover the permutations. In this paper, we propose a novel and statistically optimal spectral algorithm. Unlike the existing methods which use <inline-formula> <tex-math>${U_{j}U_{1}^{top } }_{jgeq 2}$ </tex-math></inline-formula>, ours constructs an anchor matrix <italic>M</i> by aggregating useful information from all of the block submatrices and estimates the latent permutations through <inline-formula> <tex-math>${U_{j}M^{top } }_{jgeq 1}$ </tex-math></inline-formula>. This modification overcomes a crucial limitation of the existing methods caused by the repetitive use of <inline-formula> <tex-math>$U_{1}$ </tex-math></inline-formula> and leads to an improved numerical performance. To establish the optimality of the proposed method, we carry out a fine-grained spectral analysis and obtain a sharp exponential error bound that matches the minimax rate.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 5","pages":"3779-3801"},"PeriodicalIF":2.2,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143892436","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Quantum Secure Non-Malleable Randomness Encoder and Its Applications","authors":"Rishabh Batra;Naresh Goud Boddu;Rahul Jain","doi":"10.1109/TIT.2025.3545283","DOIUrl":"https://doi.org/10.1109/TIT.2025.3545283","url":null,"abstract":"“Non-Malleable Randomness Encoder” (NMRE) was introduced by Kanukurthi et al. (2018) as a useful cryptographic primitive helpful in the construction of non-malleable codes. To the best of our knowledge, their construction is not known to be quantum secure. We provide a construction of a first rate-<inline-formula> <tex-math>$1/2$ </tex-math></inline-formula>, 2-split, quantum secure NMRE and use this in a black-box manner, to construct the following: 1) rate <inline-formula> <tex-math>$1/11$ </tex-math></inline-formula>, 3-split, quantum non-malleable code; 2) rate <inline-formula> <tex-math>$1/3$ </tex-math></inline-formula>, 3-split, quantum secure non-malleable code; and 3) rate <inline-formula> <tex-math>$1/5$ </tex-math></inline-formula>, 2-split, average case quantum secure non-malleable code.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 4","pages":"2698-2725"},"PeriodicalIF":2.2,"publicationDate":"2025-02-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143675966","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}