IEEE Transactions on Information Theory最新文献

{"title":"From Letters to Words and Back: Invertible Coding of Stationary Measures","authors":"Łukasz Dębowski","doi":"10.1109/TIT.2025.3562063","DOIUrl":"https://doi.org/10.1109/TIT.2025.3562063","url":null,"abstract":"Motivated by problems of statistical language modeling, we consider probability measures on infinite sequences over two countable alphabets of a different cardinality, such as letters and words. We introduce an invertible mapping between such measures, called the normalized transport, that preserves both stationarity and ergodicity. The normalized transport applies so called self-avoiding codes that generalize comma-separated codes and specialize bijective stationary codes. The normalized transport is also connected to the usual measure transport via underlying asymptotically mean stationary measures. It preserves the ergodic decomposition. The normalized transport and self-avoiding codes arise in the problem of successive recurrence times. In particular, we show that successive recurrence times are ergodic for an ergodic measure, which strengthens a result by Chen Moy from 1959. We also relate the entropy rates of processes linked by the normalized transport.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 6","pages":"4306-4316"},"PeriodicalIF":2.2,"publicationDate":"2025-04-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144117166","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"On the Complete Monotonicity of Rényi Entropy","authors":"Hao Wu;Lei Yu;Laigang Guo","doi":"10.1109/TIT.2025.3561384","DOIUrl":"https://doi.org/10.1109/TIT.2025.3561384","url":null,"abstract":"In this paper, we investigate the complete monotonicity of Rényi entropy along the heat flow. We confirm this property for the order of derivative up to 4, when the order of Rényi entropy is in certain regimes. We also investigate concavity of Rényi entropy power and the complete monotonicity of Tsallis entropy. We recover and slightly extend Hung’s result on the fourth-order derivative of the Tsallis entropy, and observe that the complete monotonicity holds for Tsallis entropy of order 2, which is equivalent to that the noise stability with respect to the heat semigroup is completely monotone. Based on this observation, we conjecture that the complete monotonicity holds for Tsallis entropy of all orders <inline-formula> <tex-math>$alpha in (1,2)$ </tex-math></inline-formula>. Our proofs in this paper are based on the techniques of integration-by-parts, sum-of-squares, and curve-fitting.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 7","pages":"4895-4914"},"PeriodicalIF":2.2,"publicationDate":"2025-04-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144331626","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Improved Bounds on the Size of Permutation Codes Under Kendall τ -Metric","authors":"Farzad Parvaresh;Reza Sobhani;Alireza Abdollahi;Javad Bagherian;Fatemeh Jafari;Maryam Khatami","doi":"10.1109/TIT.2025.3561119","DOIUrl":"https://doi.org/10.1109/TIT.2025.3561119","url":null,"abstract":"In order to overcome the challenges caused by flash memories and also to protect against errors related to reading information stored in DNA molecules in the shotgun sequencing method, the rank modulation method has been proposed. In the rank modulation framework, codewords are permutations. In this paper, we study the largest size <inline-formula> <tex-math>$P(n, d)$ </tex-math></inline-formula> of permutation codes of length <italic>n</i>, i.e., subsets of the set <inline-formula> <tex-math>$S_{n}$ </tex-math></inline-formula> of all permutations on <inline-formula> <tex-math>${1,ldots , n}$ </tex-math></inline-formula> with the minimum distance at least <inline-formula> <tex-math>$din left {{{1,ldots ,binom {n}{2}}}right }$ </tex-math></inline-formula> under the Kendall <inline-formula> <tex-math>$tau $ </tex-math></inline-formula>-metric. By presenting an algorithm and two theorems, we improve the known lower and upper bounds for <inline-formula> <tex-math>$P(n,d)$ </tex-math></inline-formula>. In particular, we show that <inline-formula> <tex-math>$P(n,d)=4$ </tex-math></inline-formula> for all <inline-formula> <tex-math>$ngeq 6$ </tex-math></inline-formula> and <inline-formula> <tex-math>$frac {3}{5}binom {n}{2}lt d leq frac {2}{3} binom {n}{2}$ </tex-math></inline-formula>. Additionally, we prove that for any prime number <italic>n</i> and integer <inline-formula> <tex-math>$rleq frac {n}{6}$ </tex-math></inline-formula>, <inline-formula> <tex-math>$ P(n,3)leq (n-1)!-dfrac {n-6r}{sqrt {n^{2}-8rn+20r^{2}}}sqrt {dfrac {(n-1)!}{n(n-r)!}}$ </tex-math></inline-formula>. This result greatly improves the upper bound of <inline-formula> <tex-math>$P(n,3)$ </tex-math></inline-formula> for all primes <inline-formula> <tex-math>$ngeq 37$ </tex-math></inline-formula>.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 6","pages":"4156-4166"},"PeriodicalIF":2.2,"publicationDate":"2025-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144117222","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Common Information Dimension","authors":"Xinlin Li;Osama Hanna;Suhas Diggavi;Christina Fragouli","doi":"10.1109/TIT.2025.3560674","DOIUrl":"https://doi.org/10.1109/TIT.2025.3560674","url":null,"abstract":"Quantifying the common information between random variables is a fundamental problem with a long history in information theory. Traditionally, common information is measured in number of bits and thus such measures are mostly informative when the common information is finite. However, the common information between continuous variables can be infinite; in such cases, a real-valued random vector <italic>W</i> may be needed to represent the common information, and to be used for instance for distributed simulation. In this paper, we propose the concept of Common Information Dimension (CID) and three variants. We compute the common information dimension for jointly Gaussian random vectors in a closed form. Moreover, we analytically prove, under two different formulations, that the growth rate of common information in the nearly infinite regime is determined by the common information dimension, for the case of two Gaussian vectors.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 7","pages":"4915-4938"},"PeriodicalIF":2.2,"publicationDate":"2025-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144331667","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Generalized Approximate Message-Passing for Compressed Sensing With Sublinear Sparsity","authors":"Keigo Takeuchi","doi":"10.1109/TIT.2025.3560070","DOIUrl":"https://doi.org/10.1109/TIT.2025.3560070","url":null,"abstract":"This paper addresses the reconstruction of an unknown signal vector with sublinear sparsity from generalized linear measurements. Generalized approximate message-passing (GAMP) is proposed via state evolution in the sublinear sparsity limit, where the signal dimension <italic>N</i>, measurement dimension <italic>M</i>, and signal sparsity <italic>k</i> satisfy <inline-formula> <tex-math>$log k/log Nto gamma in [0, 1$ </tex-math></inline-formula>) and <inline-formula> <tex-math>$M/{klog (N/k)}to delta $ </tex-math></inline-formula> as <italic>N</i> and <italic>k</i> tend to infinity. While the overall flow in state evolution is the same as that for linear sparsity, each proof step for inner denoising requires stronger assumptions than those for linear sparsity. The required new assumptions are proved for Bayesian inner denoising. When Bayesian outer and inner denoisers are used in GAMP, the obtained state evolution recursion is utilized to evaluate the prefactor <inline-formula> <tex-math>$delta $ </tex-math></inline-formula> in the sample complexity, called reconstruction threshold. If and only if <inline-formula> <tex-math>$delta $ </tex-math></inline-formula> is larger than the reconstruction threshold, Bayesian GAMP can achieve asymptotically exact signal reconstruction. In particular, the reconstruction threshold is finite for noisy linear measurements when the support of non-zero signal elements does not include a neighborhood of zero. As numerical examples, this paper considers linear measurements and 1-bit compressed sensing. Numerical simulations for both cases show that Bayesian GAMP outperforms existing algorithms for sublinear sparsity in terms of the sample complexity.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 6","pages":"4602-4636"},"PeriodicalIF":2.2,"publicationDate":"2025-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10963836","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144117104","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Lattice-Based Key-Value Commitment Scheme","authors":"Hideaki Miyaji;Atsuko Miyaji","doi":"10.1109/TIT.2025.3559974","DOIUrl":"https://doi.org/10.1109/TIT.2025.3559974","url":null,"abstract":"A blockchain is an important component in the design of secure distributed file systems, such as cryptocurrencies. One of the key components of the blockchain is the key-value commitment scheme, which constructs a commitment value from two inputs: a key and a value. In a conventional commitment scheme, a single user constructs a commitment value from an input value, whereas in a key-value commitment scheme, multiple users construct a commitment value from their keys and values. Both conventional and key-value commitment schemes must satisfy binding and hiding properties. The key-binding and key-hiding properties guarantee that neither the sender nor the verifier can act maliciously. The concept of a key-value commitment scheme was first proposed by Agrawal et al. in 2020 using a strong RSA assumption. Their scheme satisfies the key-binding but not key-hiding properties. In this paper, we propose two lattice-based key-value commitment schemes, <inline-formula> <tex-math>${mathsf { Insert}}text {-}{mathsf { KVC}}_{m/2,n,q,beta }$ </tex-math></inline-formula> and <inline-formula> <tex-math>${mathsf {text {KVC}}}_{m,n,q,beta }$ </tex-math></inline-formula>, that satisfy both the key-binding and the key-hiding properties. The key-binding property of both <inline-formula> <tex-math>${mathsf { Insert}}text {-}{mathsf { KVC}}_{m/2,n,q,beta }$ </tex-math></inline-formula> and <inline-formula> <tex-math>${mathsf {text {KVC}}}_{m,n,q,beta }$ </tex-math></inline-formula> are proven under the short integer solution (<inline-formula> <tex-math>${mathsf {text {SIS}}}^{infty } _{n,m,q,beta }$ </tex-math></inline-formula>) problem. The key-hiding property of both <inline-formula> <tex-math>${mathsf { Insert}}text {-}{mathsf { KVC}}_{m/2,n,q,beta }$ </tex-math></inline-formula> and <inline-formula> <tex-math>${mathsf {text {KVC}}}_{m,n,q,beta }$ </tex-math></inline-formula> are proven under the Decisional-<inline-formula> <tex-math>${mathsf {text {SIS}}}^{infty } _{n,m,q,beta }$ </tex-math></inline-formula>-form problem, which is newly defined in this paper. We demonstrate the difficulty of the Decisional-<inline-formula> <tex-math>${mathsf {text {SIS}}}^{infty } _{n,m,q,beta }$ </tex-math></inline-formula>-form problem by showing that the Decisional-<inline-formula> <tex-math>${mathsf {text {SIS}}}^{infty } _{n,m,q,beta }$ </tex-math></inline-formula>-form problem is secure when the <inline-formula> <tex-math>${mathsf {text {SIS}}}^{infty } _{n,m,q,beta }$ </tex-math></inline-formula> problem is secure. Finally, we analyze the computational costs of <inline-formula> <tex-math>${mathsf { Insert}}text {-}{mathsf { KVC}}_{m/2,n,q,beta }$ </tex-math></inline-formula> and <inline-formula> <tex-math>${mathsf {text {KVC}}}_{m,n,q,beta }$ </tex-math></inline-formula>. Our method is the first lattice-based key-value commitment scheme with proven the key-binding and the key-hiding properties.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 6","pages":"4839-4853"},"PeriodicalIF":2.2,"publicationDate":"2025-04-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10963723","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144117165","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"DNA-Correcting Codes: End-to-End Correction in DNA Storage Systems","authors":"Avital Boruchovsky;Daniella Bar-Lev;Eitan Yaakobi","doi":"10.1109/TIT.2025.3559684","DOIUrl":"https://doi.org/10.1109/TIT.2025.3559684","url":null,"abstract":"This paper introduces a new solution to DNA storage that integrates all three steps of retrieval, namely clustering, reconstruction, and error correction. <italic>DNA-correcting codes</i> are presented as a unique solution to the problem of ensuring that the output of the storage system is unique for any valid set of input strands. To this end, we introduce a novel distance metric to capture the unique behavior of the DNA storage system and provide necessary and sufficient conditions for DNA-correcting codes. We also establish bounds and constructions for these codes, including an exploration of the <inline-formula> <tex-math>$ell _{infty } $ </tex-math></inline-formula> distance applied to permutations. Here, instead of interpreting permutation elements as numerical values and assessing absolute differences, we treat them as vectors and consider the Hamming distance to better model the DNA Storage System.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 6","pages":"4214-4227"},"PeriodicalIF":2.2,"publicationDate":"2025-04-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144117331","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"On the Communication-Computation Tradeoff for Symmetric Private Linear Computation","authors":"Jinbao Zhu;Xiaohu Tang","doi":"10.1109/TIT.2025.3559736","DOIUrl":"https://doi.org/10.1109/TIT.2025.3559736","url":null,"abstract":"We consider the problem of symmetric private linear computation (SPLC) over a replicated storage system with colluding and straggler constraints. The SPLC problem allows the user to privately compute a linear combination of multiple files from a set of replicated servers, even in the presence of straggler servers that can bottleneck the entire computation. It is guaranteed that a certain number of colluding servers learn nothing about the coefficients of the linear combination, and the user must not learn any information about the files other than the desired linear computation. Unlike previous private computation literature that mainly focused on decreasing download cost from servers, we aim to establish a flexible tradeoff between communication costs and computational complexities. In particular, we propose a novel SPLC scheme under the assumption of a fixed number of stragglers. Additionally, by generalizing this SPLC scheme, we construct an adaptive SPLC scheme capable of tolerating the presence of a varying number of stragglers, even if their identities and numbers are unknown in advance. Compared to the SPLC scheme with a fixed number of stragglers, the adaptive SPLC scheme achieves a lower communication cost based on the actual number of stragglers, albeit at the cost of increased computational complexities. Both types of SPLC schemes achieve flexible performance tradeoffs and can be employed to optimize system efficiency in practice.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 6","pages":"4284-4305"},"PeriodicalIF":2.2,"publicationDate":"2025-04-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144117407","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"The Guesswork of Ordered Statistics Decoding: Guesswork Complexity and Decoder Design","authors":"Chentao Yue;Changyang She;Branka Vucetic;Yonghui Li","doi":"10.1109/TIT.2025.3559168","DOIUrl":"https://doi.org/10.1109/TIT.2025.3559168","url":null,"abstract":"This paper investigates guesswork over ordered statistics and formulates the achievable guesswork complexity of ordered statistics decoding (OSD) in binary additive white Gaussian noise (AWGN) channels. The achievable guesswork complexity is defined as the number of test error patterns (TEPs) processed by OSD immediately upon finding the correct codeword estimate. The paper first develops a new upper bound for guesswork over independent sequences by partitioning them into Hamming shells and applying Hölder’s inequality. This upper bound is then extended to ordered statistics, by constructing the conditionally independent sequences within the ordered statistics sequences. Next, we apply these bounds to characterize the statistical moments of the OSD guesswork complexity. We show that the achievable guesswork complexity of OSD at maximum decoding order can be accurately approximated by the modified Bessel function, which increases exponentially with code dimension. We also identify a guesswork complexity saturation threshold, where increasing the OSD decoding order beyond this threshold improves error performance without further raising the achievable guesswork complexity. Finally, the paper presents insights on applying these findings to enhance the design of OSD decoders.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 6","pages":"4167-4192"},"PeriodicalIF":2.2,"publicationDate":"2025-04-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144117449","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Derandomizing Codes for the Adversarial Wiretap Channel of Type II","authors":"Eric Ruzomberka;Homa Nikbakht;Christopher G. Brinton;David J. Love;H. Vincent Poor","doi":"10.1109/TIT.2025.3559440","DOIUrl":"https://doi.org/10.1109/TIT.2025.3559440","url":null,"abstract":"The adversarial wiretap channel of type II (AWTC-II) is a communication channel that can a) read a fraction of the transmitted symbols up to a given bound and b) induce both errors and erasures in a fraction of the symbols up to given bounds. The channel is controlled by an adversary who can freely choose the locations of the symbol reads, errors and erasures via a process with unbounded computational power. The AWTC-II is an extension of Ozarow’s and Wyner’s wiretap channel of type II to the adversarial channel setting. The semantic-secrecy (SS) capacity of the AWTC-II is partially known, where the best-known lower bound is non-constructive and proven via a random coding argument that uses a large number (that is, exponential in blocklength <italic>n</i>) of random bits to describe the random code. In this work, we establish a new derandomization result in which we match the best-known lower bound via a non-constructive random code that uses only <inline-formula> <tex-math>$O(n^{2})$ </tex-math></inline-formula> random bits. Unlike fully random codes, our derandomized code admits an efficient encoding algorithm and benefits from some linear structure. Our derandomization result is a novel application of <italic>random pseudolinear codes</i> – a class of non-linear codes first proposed for applications outside the AWTC-II setting, which have <italic>k</i>-wise independent codewords where <italic>k</i> is a design parameter. As the key technical tool in our analysis, we provide a novel concentration inequality for sums of random variables with limited independence, as well as a soft-covering lemma similar to that of Goldfeld, Cuff and Permuter that holds for random codes with <italic>k</i>-wise independent codewords.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 6","pages":"4686-4707"},"PeriodicalIF":2.2,"publicationDate":"2025-04-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144117156","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}