IEEE journal on selected areas in information theory最新文献

{"title":"Flow-Based Distributionally Robust Optimization","authors":"Chen Xu;Jonghyeok Lee;Xiuyuan Cheng;Yao Xie","doi":"10.1109/JSAIT.2024.3370699","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3370699","url":null,"abstract":"We present a computationally efficient framework, called \u0000<monospace>FlowDRO</monospace>\u0000, for solving flow-based distributionally robust optimization (DRO) problems with Wasserstein uncertainty sets while aiming to find continuous worst-case distribution (also called the Least Favorable Distribution, LFD) and sample from it. The requirement for LFD to be continuous is so that the algorithm can be scalable to problems with larger sample sizes and achieve better generalization capability for the induced robust algorithms. To tackle the computationally challenging infinitely dimensional optimization problem, we leverage flow-based models and continuous-time invertible transport maps between the data distribution and the target distribution and develop a Wasserstein proximal gradient flow type algorithm. In theory, we establish the equivalence of the solution by optimal transport map to the original formulation, as well as the dual form of the problem through Wasserstein calculus and Brenier theorem. In practice, we parameterize the transport maps by a sequence of neural networks progressively trained in blocks by gradient descent. We demonstrate its usage in adversarial learning, distributionally robust hypothesis testing, and a new mechanism for data-driven distribution perturbation differential privacy, where the proposed method gives strong empirical performance on high-dimensional real data.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"62-77"},"PeriodicalIF":0.0,"publicationDate":"2024-02-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140297078","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Forking Uncertainties: Reliable Prediction and Model Predictive Control With Sequence Models via Conformal Risk Control","authors":"Matteo Zecchin;Sangwoo Park;Osvaldo Simeone","doi":"10.1109/JSAIT.2024.3368229","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3368229","url":null,"abstract":"In many real-world problems, predictions are leveraged to monitor and control cyber-physical systems, demanding guarantees on the satisfaction of reliability and safety requirements. However, predictions are inherently uncertain, and managing prediction uncertainty presents significant challenges in environments characterized by complex dynamics and forking trajectories. In this work, we assume access to a pre-designed probabilistic implicit or explicit sequence model, which may have been obtained using model-based or model-free methods. We introduce probabilistic time series-conformal risk prediction (PTS-CRC), a novel post-hoc calibration procedure that operates on the predictions produced by any pre-designed probabilistic forecaster to yield reliable error bars. In contrast to existing art, PTS-CRC produces predictive sets based on an ensemble of multiple prototype trajectories sampled from the sequence model, supporting the efficient representation of forking uncertainties. Furthermore, unlike the state of the art, PTS-CRC can satisfy reliability definitions beyond coverage. This property is leveraged to devise a novel model predictive control (MPC) framework that addresses open-loop and closed-loop control problems under general average constraints on the quality or safety of the control policy. We experimentally validate the performance of PTS-CRC prediction and control by studying a number of use cases in the context of wireless networking. Across all the considered tasks, PTS-CRC predictors are shown to provide more informative predictive sets, as well as safe control policies with larger returns.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"44-61"},"PeriodicalIF":0.0,"publicationDate":"2024-02-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140297113","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Continual Mean Estimation Under User-Level Privacy","authors":"Anand Jerry George;Lekshmi Ramesh;Aditya Vikram Singh;Himanshu Tyagi","doi":"10.1109/JSAIT.2024.3366086","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3366086","url":null,"abstract":"We consider the problem of continually releasing an estimate of the population mean of a stream of samples that is user-level differentially private (DP). At each time instant, a user contributes a sample, and the users can arrive in arbitrary order. Until now these requirements of continual release and user-level privacy were considered in isolation. But, in practice, both these requirements come together as the users often contribute data repeatedly and multiple queries are made. We provide an algorithm that outputs a mean estimate at every time instant t such that the overall release is user-level \u0000<inline-formula> <tex-math>$varepsilon $ </tex-math></inline-formula>\u0000-DP and has the following error guarantee: Denoting by \u0000<inline-formula> <tex-math>$m_{t}$ </tex-math></inline-formula>\u0000 the maximum number of samples contributed by a user, as long as \u0000<inline-formula> <tex-math>$tilde {Omega }(1/varepsilon)$ </tex-math></inline-formula>\u0000 users have \u0000<inline-formula> <tex-math>$m_{t}/2$ </tex-math></inline-formula>\u0000 samples each, the error at time t is \u0000<inline-formula> <tex-math>$tilde {O}(1/sqrt {t}+sqrt {m}_{t}/tvarepsilon)$ </tex-math></inline-formula>\u0000. This is a universal error guarantee which is valid for all arrival patterns of the users. Furthermore, it (almost) matches the existing lower bounds for the single-release setting at all time instants when users have contributed equal number of samples.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"28-43"},"PeriodicalIF":0.0,"publicationDate":"2024-02-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140297133","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multi-Message Shuffled Privacy in Federated Learning","authors":"Antonious M. Girgis;Suhas Diggavi","doi":"10.1109/JSAIT.2024.3366225","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3366225","url":null,"abstract":"We study the distributed mean estimation (DME) problem under privacy and communication constraints in the local differential privacy (LDP) and multi-message shuffled (MMS) privacy frameworks. The DME has wide applications in both federated learning and analytics. We propose a communication-efficient and differentially private algorithm for DME of bounded \u0000<inline-formula> <tex-math>$ell _{2}$ </tex-math></inline-formula>\u0000-norm and \u0000<inline-formula> <tex-math>$ell _{infty }$ </tex-math></inline-formula>\u0000-norm vectors. We analyze our proposed DME schemes showing that our algorithms have order-optimal privacy-communication-performance trade-offs. Our algorithms are designed by giving unequal privacy assignments at different resolutions of the vector (through binary expansion) and appropriately combining it with coordinate sampling. These results are directly applied to give guarantees on private federated learning algorithms. We also numerically evaluate the performance of our private DME algorithms.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"12-27"},"PeriodicalIF":0.0,"publicationDate":"2024-02-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140052943","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Quickest Change Detection With Controlled Sensing","authors":"Venugopal V. Veeravalli;Georgios Fellouris;George V. Moustakides","doi":"10.1109/JSAIT.2024.3362324","DOIUrl":"https://doi.org/10.1109/JSAIT.2024.3362324","url":null,"abstract":"In the problem of quickest change detection, a change occurs at some unknown time in the distribution of a sequence of random vectors that are monitored in real time, and the goal is to detect this change as quickly as possible subject to a certain false alarm constraint. In this work we consider this problem in the presence of parametric uncertainty in the post-change regime and controlled sensing. That is, the post-change distribution contains an unknown parameter, and the distribution of each observation, before and after the change, is affected by a control action. In this context, in addition to a stopping rule that determines the time at which it is declared that the change has occurred, one also needs to determine a sequential control policy, which chooses the control action at each time based on the already collected observations. We formulate this problem mathematically using Lorden’s minimax criterion, and assuming that there are finitely many possible actions and post-change parameter values. We then propose a specific procedure for this problem that employs an adaptive CuSum statistic in which (i) the estimate of the parameter is based on a fixed number of the more recent observations, and (ii) each action is selected to maximize the Kullback-Leibler divergence of the next observation based on the current parameter estimate, apart from a small number of exploration times. We show that this procedure, which we call the Windowed Chernoff-CuSum (WCC), is first-order asymptotically optimal under Lorden’s minimax criterion, for every possible value of the unknown post-change parameter, as the mean time to false alarm goes to infinity. We also provide simulation results to illustrate the performance of the WCC procedure.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"5 ","pages":"1-11"},"PeriodicalIF":0.0,"publicationDate":"2024-02-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10423411","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140052897","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Pull or Wait: How to Optimize Query Age of Information","authors":"M. Emrullah Ildiz;Orhan T. Yavascan;Elif Uysal;O. Tugberk Kartal","doi":"10.1109/JSAIT.2023.3346308","DOIUrl":"https://doi.org/10.1109/JSAIT.2023.3346308","url":null,"abstract":"We study a pull-based status update communication model where a source node submits update packets to a channel with random transmission delay, at times requested by a remote destination node. The objective is to minimize the average query-age-of-information (QAoI), defined as the average age-of-information (AoI) measured at query instants that occur at the destination side according to a stochastic arrival process. In reference to a push-based problem formulation defined in the literature where the source decides to update or wait at will, with the objective of minimizing the time average AoI at the destination, we name this problem the Pull-or-Wait (PoW) problem. We identify the PoW problem in the case of a single query as a stochastic shortest path (SSP) problem with uncountable state and action spaces, which has not been solved in previous literature. We derive an optimal solution for this SSP problem and use it as a building block for the solution of the PoW problem under periodic query arrivals.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"4 ","pages":"794-807"},"PeriodicalIF":0.0,"publicationDate":"2023-12-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139572589","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Optimal Update Times for Stale Information Metrics Including the Age of Information","authors":"Chris Ferguson;Leonard Kleinrock","doi":"10.1109/JSAIT.2023.3344760","DOIUrl":"https://doi.org/10.1109/JSAIT.2023.3344760","url":null,"abstract":"In this paper we examine the general problem of determining when to update information that can go out-of-date. Not updating frequently enough results in poor decision making based on stale information. Updating too often results in excessive update costs. We study the tradeoff between having stale information and the cost of updating that information. We use a general model, some versions of which match an idealized version of the Age of Information (AoI) model. We first present the assumptions, and a novel methodology for solving problems of this sort. Then we solve the case where the update cost is fixed and the time-value of the information is well understood. Our results provide simple and powerful insights regarding optimal update times. We further look at cases where there are delays associated with sending a request for an update and receiving the update, cases where the update source may be stale, cases where the information cannot be used during the update process, and cases where update costs can change randomly.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"4 ","pages":"734-746"},"PeriodicalIF":0.0,"publicationDate":"2023-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10371395","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139060086","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Causal Graph Discovery From Self and Mutually Exciting Time Series","authors":"Song Wei;Yao Xie;Christopher S. Josef;Rishikesan Kamaleswaran","doi":"10.1109/JSAIT.2023.3342569","DOIUrl":"https://doi.org/10.1109/JSAIT.2023.3342569","url":null,"abstract":"We present a generalized linear structural causal model, coupled with a novel data-adaptive linear regularization, to recover causal directed acyclic graphs (DAGs) from time series. By leveraging a recently developed stochastic monotone Variational Inequality (VI) formulation, we cast the causal discovery problem as a general convex optimization. Furthermore, we develop a non-asymptotic recovery guarantee and quantifiable uncertainty by solving a linear program to establish confidence intervals for a wide range of non-linear monotone link functions. We validate our theoretical results and show the competitive performance of our method via extensive numerical experiments. Most importantly, we demonstrate the effectiveness of our approach in recovering highly interpretable causal DAGs over Sepsis Associated Derangements (SADs) while achieving comparable prediction performance to powerful “black-box” models such as XGBoost.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"4 ","pages":"747-761"},"PeriodicalIF":0.0,"publicationDate":"2023-12-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139090401","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Toward Goal-Oriented Semantic Communications: AoII Analysis of Coded Status Update System Under FBL Regime","authors":"Siqi Meng;Shaohua Wu;Aimin Li;Qinyu Zhang","doi":"10.1109/JSAIT.2023.3344586","DOIUrl":"https://doi.org/10.1109/JSAIT.2023.3344586","url":null,"abstract":"In the past decade, the emergence of beyond fifth generation (B5G) wireless networks has necessitated the timely updating of system states in Internet of Things (IoT) and cyber-physical systems, where Age of Information (AoI) has been a well-concentrated metric. However, the content-agnostic nature of AoI reflects its limitation of characterizing the significance of status update messages, which induces various variants for AoI including Age of Incorrect Information (AoII). AoII is a goal-oriented significance (etymological meaning of “semantics”) metric that could overcome such shortcomings, and thus analyzing AoII performance can be a potential approach of realizing semantic communications. Nevertheless, AoII analysis of practical coded status update system under finite blocklength (FBL) regime is still in its nascent stages. To the best of our knowledge, our study represents the first analysis of AoII for FBL regime. We explicitly obtain the average AoII expressions for different transmission schemes including Automatic Repeat reQuest (ARQ), Hybrid ARQ (HARQ), and non-ARQ transmission schemes. Moreover, we theoretically prove that non-ARQ scheme outperforms ARQ schemes in terms of AoII, and numerically compare AoII performance between non-ARQ and HARQ schemes by formulating and solving the AoII-optimal block assignment problem. Extensive simulation results show the superiority of AoII-optimal transmission schemes.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"4 ","pages":"718-733"},"PeriodicalIF":0.0,"publicationDate":"2023-12-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139050591","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Causal Semantic Communication for Digital Twins: A Generalizable Imitation Learning Approach","authors":"Christo Kurisummoottil Thomas;Walid Saad;Yong Xiao","doi":"10.1109/JSAIT.2023.3336538","DOIUrl":"https://doi.org/10.1109/JSAIT.2023.3336538","url":null,"abstract":"A digital twin (DT) leverages a virtual representation of the physical world, along with communication (e.g., 6G), computing (e.g., edge computing), and artificial intelligence (AI) technologies to enable many connected intelligence services. In order to handle the large amounts of network data based on digital twins (DTs), wireless systems can exploit the paradigm of semantic communication (SC) for facilitating informed decision-making under strict communication constraints by utilizing AI techniques such as causal reasoning. In this paper, a novel framework called causal semantic communication (CSC) is proposed for DT-based wireless systems. The CSC system is posed as an imitation learning (IL) problem, where the transmitter, with access to optimal network control policies using a DT, teaches the receiver using SC over a bandwidth-limited wireless channel how to improve its knowledge to perform optimal control actions. The causal structure in the transmitter’s data is extracted using novel approaches from the framework of deep end-to-end causal inference, thereby enabling the creation of a semantic representation that is causally invariant, which in turn helps generalize the learned knowledge of the system to new and unseen situations. The CSC decoder at the receiver is designed to extract and estimate semantic information while ensuring high semantic reliability. The receiver control policies, semantic decoder, and causal inference are formulated as a bi-level optimization problem within a variational inference framework. This problem is solved using a novel concept called network state models, inspired from world models in generative AI, that faithfully represents the environment dynamics leading to data generation. Furthermore, the proposed framework includes an analytical characterization of the performance gap that results from employing a suboptimal policy learned by the receiver that uses the transmitted semantic information to construct a model of the physical environment. The CSC system utilizes two concepts, namely the integrated information theory principle in the theory of consciousness and the abstract cell complex concept in topology, to precisely express the information content conveyed by the causal states and their relationships. Through this analysis, novel formulations of semantic information, semantic reliability, distortion, and similarity metrics are proposed, which extend beyond Shannon’s concept of uncertainty. Simulation results demonstrate that the proposed CSC system outperforms conventional wireless and state-of-the-art SC systems by achieving better semantic reliability with reduced bits and enabling better control policies over time thanks to the generative AI architecture.","PeriodicalId":73295,"journal":{"name":"IEEE journal on selected areas in information theory","volume":"4 ","pages":"698-717"},"PeriodicalIF":0.0,"publicationDate":"2023-11-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138822192","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}