{"title":"Generalized Measures of Anticipation and Responsivity in Online Language Processing","authors":"Mario Giulianelli, Andreas Opedal, Ryan Cotterell","doi":"arxiv-2409.10728","DOIUrl":"https://doi.org/arxiv-2409.10728","url":null,"abstract":"We introduce a generalization of classic information-theoretic measures of\u0000predictive uncertainty in online language processing, based on the simulation\u0000of expected continuations of incremental linguistic contexts. Our framework\u0000provides a formal definition of anticipatory and responsive measures, and it\u0000equips experimenters with the tools to define new, more expressive measures\u0000beyond standard next-symbol entropy and surprisal. While extracting these\u0000standard quantities from language models is convenient, we demonstrate that\u0000using Monte Carlo simulation to estimate alternative responsive and\u0000anticipatory measures pays off empirically: New special cases of our\u0000generalized formula exhibit enhanced predictive power compared to surprisal for\u0000human cloze completion probability as well as ELAN, LAN, and N400 amplitudes,\u0000and greater complementarity with surprisal in predicting reading times.","PeriodicalId":501082,"journal":{"name":"arXiv - MATH - Information Theory","volume":"8 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142268477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Second Generalized Covering Radius of Binary Primitive Double-Error-Correcting BCH Codes","authors":"Lev Yohananov, Moshe Schwartz","doi":"arxiv-2409.10420","DOIUrl":"https://doi.org/arxiv-2409.10420","url":null,"abstract":"We completely determine the second covering radius for binary primitive\u0000double-error-correcting BCH codes. As part of this process, we provide a lower\u0000bound on the second covering radius for binary primitive BCH codes correcting\u0000more than two errors.","PeriodicalId":501082,"journal":{"name":"arXiv - MATH - Information Theory","volume":"34 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142268479","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Wenyu Wang, Minhao Zhu, Kaiming Shen, Zhaorui Wang, Shuguang Cui
{"title":"Power Allocation for Finite-Blocklength IR-HARQ","authors":"Wenyu Wang, Minhao Zhu, Kaiming Shen, Zhaorui Wang, Shuguang Cui","doi":"arxiv-2409.09780","DOIUrl":"https://doi.org/arxiv-2409.09780","url":null,"abstract":"This letter concerns the power allocation across the multiple transmission\u0000rounds under the Incremental Redundancy Hybrid Automatic Repeat reQuest\u0000(IR-HARQ) policy, in pursuit of an energy-efficient way of fulfilling the\u0000outage probability target in the finite-blocklength regime. We start by showing\u0000that the optimization objective and the constraints of the above power\u0000allocation problem all depend upon the outage probability. The main challenge\u0000then lies in the fact that the outage probability cannot be written\u0000analytically in terms of the power variables. To sidestep this difficulty, we\u0000propose a novel upper bound on the outage probability in the finite-blocklength\u0000regime, which is much tighter than the existing ones from the literature. Most\u0000importantly, by using this upper bound to approximate the outage probability,\u0000we can recast the original intractable power allocation problem into a\u0000geometric programming (GP) form--which can be efficiently solved by the\u0000standard method.","PeriodicalId":501082,"journal":{"name":"arXiv - MATH - Information Theory","volume":"50 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142268481","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Simple Study on the Optimality of Hybrid NOMA","authors":"Zhiguo Ding","doi":"arxiv-2409.09654","DOIUrl":"https://doi.org/arxiv-2409.09654","url":null,"abstract":"The key idea of hybrid non-orthogonal multiple access (NOMA) is to allow\u0000users to use the bandwidth resources to which they cannot have access in\u0000orthogonal multiple access (OMA) based legacy networks while still guaranteeing\u0000its compatibility with the legacy network. However, in a conventional hybrid\u0000NOMA network, some users have access to more bandwidth resources than others,\u0000which leads to a potential performance loss. So what if the users can access\u0000the same amount of bandwidth resources? This letter focuses on a simple\u0000two-user scenario, and develops analytical and simulation results to reveal\u0000that for this considered scenario, conventional hybrid NOMA is still an optimal\u0000transmission strategy.","PeriodicalId":501082,"journal":{"name":"arXiv - MATH - Information Theory","volume":"207 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142268483","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mengmeng Ren, Li Qiao, Long Yang, Zhen Gao, Jian Chen, Mahdi Boloursaz Mashhadi, Pei Xiao, Rahim Tafazolli, Mehdi Bennis
{"title":"Generative Semantic Communication via Textual Prompts: Latency Performance Tradeoffs","authors":"Mengmeng Ren, Li Qiao, Long Yang, Zhen Gao, Jian Chen, Mahdi Boloursaz Mashhadi, Pei Xiao, Rahim Tafazolli, Mehdi Bennis","doi":"arxiv-2409.09715","DOIUrl":"https://doi.org/arxiv-2409.09715","url":null,"abstract":"This paper develops an edge-device collaborative Generative Semantic\u0000Communications (Gen SemCom) framework leveraging pre-trained Multi-modal/Vision\u0000Language Models (M/VLMs) for ultra-low-rate semantic communication via textual\u0000prompts. The proposed framework optimizes the use of M/VLMs on the wireless\u0000edge/device to generate high-fidelity textual prompts through visual\u0000captioning/question answering, which are then transmitted over a wireless\u0000channel for SemCom. Specifically, we develop a multi-user Gen SemCom framework\u0000using pre-trained M/VLMs, and formulate a joint optimization problem of prompt\u0000generation offloading, communication and computation resource allocation to\u0000minimize the latency and maximize the resulting semantic quality. Due to the\u0000nonconvex nature of the problem with highly coupled discrete and continuous\u0000variables, we decompose it as a two-level problem and propose a low-complexity\u0000swap/leaving/joining (SLJ)-based matching algorithm. Simulation results\u0000demonstrate significant performance improvements over the conventional\u0000semanticunaware/non-collaborative offloading benchmarks.","PeriodicalId":501082,"journal":{"name":"arXiv - MATH - Information Theory","volume":"21 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142268482","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pragya GuptaDepartment of Mathematics Indian Institute of Technology Kharagpur, Debjani ChakrabortyDepartment of Mathematics Indian Institute of Technology Kharagpur, Debashree GuhaSchool of Medical Science and Technology Indian Institute of Technology Kharagpur
{"title":"Inter Observer Variability Assessment through Ordered Weighted Belief Divergence Measure in MAGDM Application to the Ensemble Classifier Feature Fusion","authors":"Pragya GuptaDepartment of Mathematics Indian Institute of Technology Kharagpur, Debjani ChakrabortyDepartment of Mathematics Indian Institute of Technology Kharagpur, Debashree GuhaSchool of Medical Science and Technology Indian Institute of Technology Kharagpur","doi":"arxiv-2409.08450","DOIUrl":"https://doi.org/arxiv-2409.08450","url":null,"abstract":"A large number of multi-attribute group decisionmaking (MAGDM) have been\u0000widely introduced to obtain consensus results. However, most of the\u0000methodologies ignore the conflict among the experts opinions and only consider\u0000equal or variable priorities of them. Therefore, this study aims to propose an\u0000Evidential MAGDM method by assessing the inter-observational variability and\u0000handling uncertainty that emerges between the experts. The proposed framework\u0000has fourfold contributions. First, the basic probability assignment (BPA)\u0000generation method is introduced to consider the inherent characteristics of\u0000each alternative by computing the degree of belief. Second, the ordered\u0000weighted belief and plausibility measure is constructed to capture the overall\u0000intrinsic information of the alternative by assessing the inter-observational\u0000variability and addressing the conflicts emerging between the group of experts.\u0000An ordered weighted belief divergence measure is constructed to acquire the\u0000weighted support for each group of experts to obtain the final preference\u0000relationship. Finally, we have shown an illustrative example of the proposed\u0000Evidential MAGDM framework. Further, we have analyzed the interpretation of\u0000Evidential MAGDM in the real-world application for ensemble classifier feature\u0000fusion to diagnose retinal disorders using optical coherence tomography images.","PeriodicalId":501082,"journal":{"name":"arXiv - MATH - Information Theory","volume":"20 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142268488","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Learning Short Codes for Fading Channels with No or Receiver-Only Channel State Information","authors":"Rishabh Sharad Pomaje, Rajshekhar V Bhat","doi":"arxiv-2409.08581","DOIUrl":"https://doi.org/arxiv-2409.08581","url":null,"abstract":"In next-generation wireless networks, low latency often necessitates\u0000short-length codewords that either do not use channel state information (CSI)\u0000or rely solely on CSI at the receiver (CSIR). Gaussian codes that achieve\u0000capacity for AWGN channels may be unsuitable for these no-CSI and CSIR-only\u0000cases. In this work, we design short-length codewords for these cases using an\u0000autoencoder architecture. From the designed codes, we observe the following: In\u0000the no-CSI case, the learned codes are mutually orthogonal when the\u0000distribution of the real and imaginary parts of the fading random variable has\u0000support over the entire real line. However, when the support is limited to the\u0000non-negative real line, the codes are not mutually orthogonal. For the\u0000CSIR-only case, deep learning-based codes designed for AWGN channels perform\u0000worse in fading channels with optimal coherent detection compared to codes\u0000specifically designed for fading channels with CSIR, where the autoencoder\u0000jointly learns encoding, coherent combining, and decoding. In both no-CSI and\u0000CSIR-only cases, the codes perform at least as well as or better than classical\u0000codes of the same block length.","PeriodicalId":501082,"journal":{"name":"arXiv - MATH - Information Theory","volume":"8 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142268485","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hongjiang Lei, Mingxu Yang, Ki-Hong Park, Gaofeng Pan
{"title":"Secure Offloading in NOMA-Aided Aerial MEC Systems Based on Deep Reinforcement Learning","authors":"Hongjiang Lei, Mingxu Yang, Ki-Hong Park, Gaofeng Pan","doi":"arxiv-2409.08579","DOIUrl":"https://doi.org/arxiv-2409.08579","url":null,"abstract":"Mobile edge computing (MEC) technology can reduce user latency and energy\u0000consumption by offloading computationally intensive tasks to the edge servers.\u0000Unmanned aerial vehicles (UAVs) and non-orthogonal multiple access (NOMA)\u0000technology enable the MEC networks to provide offloaded computing services for\u0000massively accessed terrestrial users conveniently. However, the broadcast\u0000nature of signal propagation in NOMA-based UAV-MEC networks makes it vulnerable\u0000to eavesdropping by malicious eavesdroppers. In this work, a secure offload\u0000scheme is proposed for NOMA-based UAV-MEC systems with the existence of an\u0000aerial eavesdropper. The long-term average network computational cost is\u0000minimized by jointly designing the UAV's trajectory, the terrestrial users'\u0000transmit power, and computational frequency while ensuring the security of\u0000users' offloaded data. Due to the eavesdropper's location uncertainty, the\u0000worst-case security scenario is considered through the estimated eavesdropping\u0000range. Due to the high-dimensional continuous action space, the deep\u0000deterministic policy gradient algorithm is utilized to solve the non-convex\u0000optimization problem. Simulation results validate the effectiveness of the\u0000proposed scheme.","PeriodicalId":501082,"journal":{"name":"arXiv - MATH - Information Theory","volume":"17 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142268487","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Deep Learning-based Codes for Wiretap Fading Channels","authors":"Daniel Seifert, Onur Günlü, Rafael F. Schaefer","doi":"arxiv-2409.08786","DOIUrl":"https://doi.org/arxiv-2409.08786","url":null,"abstract":"The wiretap channel is a well-studied problem in the physical layer security\u0000(PLS) literature. Although it is proven that the decoding error probability and\u0000information leakage can be made arbitrarily small in the asymptotic regime,\u0000further research on finite-blocklength codes is required on the path towards\u0000practical, secure communications systems. This work provides the first\u0000experimental characterization of a deep learning-based, finite-blocklength code\u0000construction for multi-tap fading wiretap channels without channel state\u0000information (CSI). In addition to the evaluation of the average probability of\u0000error and information leakage, we illustrate the influence of (i) the number of\u0000fading taps, (ii) differing variances of the fading coefficients and (iii) the\u0000seed selection for the hash function-based security layer.","PeriodicalId":501082,"journal":{"name":"arXiv - MATH - Information Theory","volume":"21 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142268486","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Generalized Nearest Neighbor Decoding: General Input Constellation and a Case Study of Interference Suppression","authors":"Shuqin Pang, Wenyi Zhang","doi":"arxiv-2409.08826","DOIUrl":"https://doi.org/arxiv-2409.08826","url":null,"abstract":"In this work, generalized nearest neighbor decoding (GNND), a recently\u0000proposed receiver architecture, is studied for channels under general input\u0000constellations, and multiuser uplink interference suppression is employed as a\u0000case study for demonstrating its potential. In essence, GNND generalizes the\u0000wellknown nearest neighbor decoding, by introducing a symbol-level memoryless\u0000processing step, which can be rendered seamlessly compatible with Gaussian\u0000channel-based decoders. First, criteria of the optimal GNND are derived for\u0000general input constellations, expressed in the form of conditional moments\u0000matching, thereby generalizing the prior work which has been confined to\u0000Gaussian input. Then, the optimal GNND is applied to the use case of multiuser\u0000uplink, for which the optimal GNND is shown to be capable of achieving\u0000information rates nearly identical to the channel mutual information. By\u0000contrast, the commonly used channel linearization (CL) approach incurs a\u0000noticeable rate loss. A coded modulation scheme is subsequently developed,\u0000aiming at implementing GNND using off-the-shelf channel codes, without\u0000requiring iterative message passing between demodulator and decoder. Through\u0000numerical experiments it is validated that the developed scheme significantly\u0000outperforms the CL-based scheme.","PeriodicalId":501082,"journal":{"name":"arXiv - MATH - Information Theory","volume":"33 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142268484","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}