{"title":"Subspace and DOA Estimation Under Coarse Quantization","authors":"Sjoerd Dirksen;Weilin Li;Johannes Maly","doi":"10.1109/TIT.2025.3598702","DOIUrl":"https://doi.org/10.1109/TIT.2025.3598702","url":null,"abstract":"We study direction-of-arrival (DOA) estimation from coarsely quantized data. We focus on a two-step approach which first estimates the signal subspace via covariance estimation and then extracts DOA angles by the ESPRIT algorithm. In particular, we analyze two stochastic quantization schemes which use dithering: a one-bit quantizer combined with rectangular dither and a multi-bit quantizer with triangular dither. For each quantizer, we derive rigorous high probability bounds for the distances between the true and estimated signal subspaces and DOA angles. Using our analysis, we identify scenarios in which subspace and DOA estimation via triangular dithering qualitatively outperforms rectangular dithering. We verify in numerical simulations that our estimates are optimal in their dependence on the smallest non-zero eigenvalue of the target matrix. The resulting subspace estimation guarantees are equally applicable in the analysis of other spectral estimation algorithms and related problems.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 10","pages":"8149-8168"},"PeriodicalIF":2.9,"publicationDate":"2025-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145110310","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Sample-Efficient Reinforcement Learning From Human Feedback via Information-Directed Sampling","authors":"Han Qi;Haochen Yang;Qiaosheng Zhang;Zhuoran Yang","doi":"10.1109/TIT.2025.3598296","DOIUrl":"https://doi.org/10.1109/TIT.2025.3598296","url":null,"abstract":"We study the problem of reinforcement learning from human feedback (RLHF), a critical problem in training large language models, from a theoretical perspective. Our main contribution is the design of novel sample-efficient RLHF algorithms based on information-directed sampling (IDS), an online decision-making principle inspired by information theory. Our algorithms maximize the sum of the value function and a mutual information term that encourages exploration of the unknown environment (which quantifies the information gained about the environment through observed human feedback data). To tackle the challenge of large state spaces and improve sample efficiency, we construct a simplified <italic>surrogate environment</i> and introduce a novel distance measure (named the <inline-formula> <tex-math>$ell _{g}$ </tex-math></inline-formula><italic>-distance</i>), enabling our IDS-based algorithm to achieve a Bayesian regret upper bound of order <inline-formula> <tex-math>$O(H^{3/2}sqrt {log (K(epsilon)) T})$ </tex-math></inline-formula>, where <italic>H</i> is the episode length, <italic>T</i> is the number of episode and <inline-formula> <tex-math>$K(epsilon)$ </tex-math></inline-formula> is related to the covering number of the environment. Specializing to the tabular settings, this regret bound is of order <inline-formula> <tex-math>$tilde {O}(H^{2}sqrt {SAT})$ </tex-math></inline-formula>, where <italic>S</i> and <italic>A</i> are the numbers of states and actions. Finally, we propose an Approximate-IDS algorithm that is computationally more efficient while maintaining nearly the same sample efficiency. The design principle of this approximate algorithm is not only effective in RLHF settings but also applicable to the standard RL framework. Moreover, our work showcases the value of information theory in reinforcement learning and in the training of large language models.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 10","pages":"7942-7958"},"PeriodicalIF":2.9,"publicationDate":"2025-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145110261","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Bregman-Divergence-Based Arimoto-Blahut Algorithm","authors":"Masahito Hayashi","doi":"10.1109/TIT.2025.3597943","DOIUrl":"https://doi.org/10.1109/TIT.2025.3597943","url":null,"abstract":"We generalize the generalized Arimoto-Blahut algorithm to a general function defined over Bregman-divergence system. In existing methods, when linear constraints are imposed, each iteration needs to solve a convex minimization. Exploiting our obtained algorithm, we propose a minimization-free-iteration algorithm. This algorithm can be applied to classical and quantum rate-distortion theory. We numerically apply our method to the derivation of the optimal conditional distribution in the rate-distortion theory.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 10","pages":"7788-7801"},"PeriodicalIF":2.9,"publicationDate":"2025-08-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145110325","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"On Non-Interactive Simulation of Distributed Sources With Finite Alphabets","authors":"Hojat Allah Salehi;Farhad Shirani","doi":"10.1109/TIT.2025.3597546","DOIUrl":"https://doi.org/10.1109/TIT.2025.3597546","url":null,"abstract":"This work presents a Fourier analysis framework for the non-interactive source simulation (NISS) problem. Two distributed agents observe a pair of sequences <inline-formula> <tex-math>$X^{d}$ </tex-math></inline-formula> and <inline-formula> <tex-math>$Y^{d}$ </tex-math></inline-formula> drawn according to a joint distribution <inline-formula> <tex-math>$P_{X^{d}Y^{d}}$ </tex-math></inline-formula>. The agents aim to generate outputs <inline-formula> <tex-math>$U=f_{d}(X^{d})$ </tex-math></inline-formula> and <inline-formula> <tex-math>$V=g_{d}(Y^{d})$ </tex-math></inline-formula> with a joint distribution sufficiently close in total variation to a target distribution <inline-formula> <tex-math>$Q_{UV}$ </tex-math></inline-formula>. Existing works have shown that the NISS problem with finite-alphabet outputs is decidable. For the binary-output NISS, an upper-bound to the input complexity was derived which is <inline-formula> <tex-math>$Oleft ({{exp mathrm {poly}left ({{frac {1}{epsilon }}}right)}}right)$ </tex-math></inline-formula>. In this work, the input complexity and algorithm design are addressed in several classes of NISS scenarios. For binary-output NISS scenarios with doubly-symmetric binary inputs, it is shown that the input complexity is <inline-formula> <tex-math>$Theta left ({{log {frac {1}{epsilon }}}}right)$ </tex-math></inline-formula>, thus providing a super-exponential improvement in input complexity. An explicit characterization of the simulating pair of functions is provided. For general finite-input scenarios, a constructive algorithm is introduced that explicitly finds the simulating functions <inline-formula> <tex-math>$(f_{d}(X^{d}),g_{d}(Y^{d}))$ </tex-math></inline-formula>. The approach relies on a novel Fourier analysis framework. Various numerical simulations of NISS scenarios with IID inputs are provided. Furthermore, to illustrate the general applicability of the Fourier framework, several examples with non-IID inputs, including entanglement-assisted NISS and NISS with Markovian inputs are provided.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 10","pages":"8048-8079"},"PeriodicalIF":2.9,"publicationDate":"2025-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145110322","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Properties of Algorithmic Information Distance","authors":"Marcus Hutter","doi":"10.1109/TIT.2025.3597092","DOIUrl":"https://doi.org/10.1109/TIT.2025.3597092","url":null,"abstract":"The domain-independent universal Normalized Information Distance based on Kolmogorov complexity has been (in approximate form) successfully applied to a variety of difficult clustering problems. In this paper we investigate theoretical properties of the un-normalized algorithmic information distance <inline-formula> <tex-math>$d_{K}$ </tex-math></inline-formula>. The main question we are asking in this work is what properties this curious distance has, besides being a metric. We show that many (in)finite-dimensional spaces can(not) be isometrically scale-embedded into the space of finite strings with metric <inline-formula> <tex-math>$d_{K}$ </tex-math></inline-formula>. We also show that <inline-formula> <tex-math>$d_{K}$ </tex-math></inline-formula> is not an Euclidean distance, but any finite set of points in Euclidean space can be scale-embedded into <inline-formula> <tex-math>$({0,1}^{*},d_{K})$ </tex-math></inline-formula>. A major contribution is the development of the necessary framework and tools for finding more (interesting) properties of <inline-formula> <tex-math>$d_{K}$ </tex-math></inline-formula> in future, and to state several open problems.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 10","pages":"7540-7554"},"PeriodicalIF":2.9,"publicationDate":"2025-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145110241","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Analysis of Functions of Low Differential Uniformity in Characteristic 2: A New Approach (I)","authors":"Nurdagül Anbar;Tekgül Kalaycı;Alev Topuzoğlu","doi":"10.1109/TIT.2025.3597162","DOIUrl":"https://doi.org/10.1109/TIT.2025.3597162","url":null,"abstract":"We introduce a new concept, the <italic>APN-defect</i>, which can be thought of as measuring the distance of a given function <inline-formula> <tex-math>$G:mathbb {F}_{2^{n}} rightarrow mathbb {F}_{2^{n}}$ </tex-math></inline-formula> to the set of almost perfect nonlinear (APN) functions. This concept is motivated by the detailed analysis of the differential behaviour of non-APN functions (of low differential uniformity) <italic>G</i> using the so-called <italic>difference squares</i>. Indeed, the insight into some structural qualities of S-boxes provided by this new approach is particularly useful in the light of recent refinements of differential cryptanalysis. We describe the relations between the APN-defect and other current concepts of similar nature. Values of APN-defect for several classes of functions of interest, including Dembowski-Ostrom polynomials are given. This enables one to identify the <italic>quasi-APN</i> ones, i.e., those with favourable differential behavior. The difference square corresponding to a modification of the inverse function is determined, its APN-defect depending on <italic>n</i> is evaluated, the partial quadruple system associated to it is described, and the implications are discussed. In the forthcoming second part of this work we further examine the APN-defect of modifications of the inverse function and address some questions concerning CCZ-equivalence. We also study modifications of classes of functions of low differential uniformity over infinitely many extensions of <inline-formula> <tex-math>$mathbb {F}_{2^{n}}$ </tex-math></inline-formula> and present quantitative results on their differential behaviour.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 10","pages":"8002-8016"},"PeriodicalIF":2.9,"publicationDate":"2025-08-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145110319","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Periodic Gaussian Process Controlled B-Spline for Scalable Modeling of Irregularly Spaced Signals","authors":"Yongxiang Li;Yuanyuan Li;Di Wang","doi":"10.1109/TIT.2025.3595144","DOIUrl":"https://doi.org/10.1109/TIT.2025.3595144","url":null,"abstract":"Existing periodic Gaussian process (PGP) modeling methods rely on the regularly-spaced-signal assumption (i.e., signals are evenly spaced) and the integer-period assumption for the sake of computational feasibility. However, such an assumption prevents conventional efficient modeling approaches from working properly on irregularly (unevenly) spaced signals, such as evenly spaced signals with missing data. Moreover, without the integer-period assumption, it is computationally prohibitive to accurately search the decimal period of PGP due to the severe non-convexity of its likelihood function. To address these issues, this study proposes a PGP-controlled B-spline for scalable modeling of irregularly spaced signals with a decimal period. The proposed model integrates PGP with B-spline basis functions, allowing for nonlinear and nonparametric modeling of periodic signals. An explore-exploit optimization is developed to overcome the non-convexity of the likelihood, enabling effective and efficient decimal period estimation. The proposed PGP modeling approach has a linear time complexity. Asymptotic properties of the proposed method are studied, which shed light on the period estimation of other PGP models. Simulation and real case studies are conducted to demonstrate the superiority of the proposed method.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 10","pages":"7842-7855"},"PeriodicalIF":2.9,"publicationDate":"2025-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145110191","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Daniel Escudero;Cheng Hong;Hongqing Liu;Chaoping Xing;Chen Yuan
{"title":"Degree-D Reverse Multiplication-Friendly Embeddings","authors":"Daniel Escudero;Cheng Hong;Hongqing Liu;Chaoping Xing;Chen Yuan","doi":"10.1109/TIT.2025.3596305","DOIUrl":"https://doi.org/10.1109/TIT.2025.3596305","url":null,"abstract":"Reverse multiplication-friendly embeddings have played a crucial role in secure multiparty computation and zero-knowledge proofs. In this work, we generalize the notion of RMFEs to <italic>degree-D RMFEs</i>. We present a general construction of degree-<italic>D</i> RMFEs by generalizing the ideas on algebraic geometry used to construct traditional degree-2 RMFEs. Furthermore, our theory is given in a unified manner for general Galois rings, which include both rings of the form <inline-formula> <tex-math>$mathbb {Z}_{p^{k}}$ </tex-math></inline-formula> and fields like <inline-formula> <tex-math>$mathbb {F}_{p^{k}}$ </tex-math></inline-formula>, which have been treated separately in prior works. We present multiple concrete sets of parameters for degree-<italic>D</i> RMFEs (including <inline-formula> <tex-math>$D=2$ </tex-math></inline-formula>), which can be useful for future works. In the recent work of (Cheon & Lee, Eurocrypt’22), the concept of a <italic>degree-D packing method</i> was formally introduced, which captures the idea of embedding multiple elements of a smaller ring into a larger ring. We show that the generalized notion of RMFEs to <italic>degree-D RMFEs</i> which, in spite of being “more algebraic” than packing methods, turn out to be essentially equivalent. Thus, our constructions of degree-<italic>D</i> RMFEs are also degree-<italic>D</i> packing methods.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 10","pages":"7990-8001"},"PeriodicalIF":2.9,"publicationDate":"2025-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145110243","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Higher Grassmann Codes III: Quantum Variants","authors":"Mahir Bilen Can;Roy Joshua","doi":"10.1109/TIT.2025.3596479","DOIUrl":"https://doi.org/10.1109/TIT.2025.3596479","url":null,"abstract":"We show that it is possible to break up Higher Grassmann codes we constructed in earlier work to a sequence of affine Reed-Muller codes so that various operations can be reduced to performing these operations for the component affine Reed-Muller codes. Then we consider quantum codes produced from a pair of Higher Grassmann codes and discuss also their implementation aspects in detail.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 10","pages":"7585-7594"},"PeriodicalIF":2.9,"publicationDate":"2025-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145110253","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Distributed Semi-Supervised Inference for Generalized Linear Models With Block-Wise Missing Covariates","authors":"Ziyuan Wang;Jin Liu;Jun Shao;Heng Lian;Lei Wang","doi":"10.1109/TIT.2025.3596304","DOIUrl":"https://doi.org/10.1109/TIT.2025.3596304","url":null,"abstract":"For a relatively small labeled dataset from high-dimensional generalized linear models with block-wise missing covariates and a large unlabeled dataset, we utilize a model-assisted approach in the labeled dataset to address the issue of block-wise missing covariates and then integrate the unlabeled data to construct estimation equations for the coefficients without any imputation. A lasso-penalized semi-supervised estimator is obtained, and then its debiased estimator is proposed to establish asymptotic normality/confidence intervals. When the labeled data are distributed in multiple machines independently and only some machines have unlabeled data, we further propose a distributed debiased semi-supervised estimator for estimation and inference. The finite sample performance of our proposed two estimators is studied through simulations and further illustrated with a breast cancer dataset.","PeriodicalId":13494,"journal":{"name":"IEEE Transactions on Information Theory","volume":"71 10","pages":"7815-7841"},"PeriodicalIF":2.9,"publicationDate":"2025-08-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145110311","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}