IF 2.1 3区物理与天体物理

Entropy Pub Date : 2025-04-03 DOI: 10.3390/e27040383

Jonathan Dexter, Ian J Ford

{"title":"Stochastic Entropy Production for Classical and Quantum Dynamical Systems with Restricted Diffusion.","authors":"Jonathan Dexter, Ian J Ford","doi":"10.3390/e27040383","DOIUrl":"https://doi.org/10.3390/e27040383","url":null,"abstract":"Modeling the evolution of a system using stochastic dynamics typically implies increasing subjective uncertainty in the adopted state of the system and its environment as time progresses, and stochastic entropy production has been developed as a measure of this change. In some situations, the evolution of stochastic entropy production can be described using an Itô process, but mathematical difficulties can emerge if diffusion in the system phase space happens to be restricted to a subspace of a lower dimension. This situation can arise if there are constants of the motion, for example, or more generally when there are functions of the coordinates that evolve without noise. More simply, difficulties can emerge if there are more coordinates than there are independent noises. We show how the problem of computing the stochastic entropy production in such a situation can be overcome. We illustrate the approach using a simple case of diffusion on an ellipse. We go on to consider an open three-level quantum system modeled within a framework of Markovian quantum state diffusion. We show how a nonequilibrium stationary state of the system, with a constant mean rate of stochastic entropy production, can be established under suitable environmental couplings.","PeriodicalId":11694,"journal":{"name":"Entropy","volume":"27 4","pages":""},"PeriodicalIF":2.1,"publicationDate":"2025-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12025737/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143976671","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Complexity Analysis of Environmental Time Series. 环境时间序列的复杂性分析。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2025-04-03 DOI: 10.3390/e27040381

Holger Lange, Michael Hauhs

{"title":"Complexity Analysis of Environmental Time Series.","authors":"Holger Lange, Michael Hauhs","doi":"10.3390/e27040381","DOIUrl":"https://doi.org/10.3390/e27040381","url":null,"abstract":"Small, forested catchments are prototypes of terrestrial ecosystems and have been studied in several disciplines of environmental science over several decades. Time series of water and matter fluxes and nutrient concentrations from these systems exhibit a bewildering diversity of spatiotemporal patterns, indicating the intricate nature of processes acting on a large range of time scales. Nonlinear dynamics is an obvious framework to investigate catchment time series. We analyzed selected long-term data from three headwater catchments in the Bramke valley, Harz mountains, Lower Saxony in Germany at common biweekly resolution for the period 1991 to 2023. For every time series, we performed gap filling, detrending, and removal of the annual cycle using singular system analysis (SSA), and then calculated metrics based on ordinal pattern statistics: the permutation entropy, permutation complexity, and Fisher information, as well as their generalized versions (q-entropy and α-entropy). Further, the position of each variable in Tarnopolski diagrams is displayed and compared to reference stochastic processes, like fractional Brownian motion, fractional Gaussian noise, and β noise. Still another way of distinguishing deterministic chaos and structured noise, and quantifying the latter, is provided by the complexity from ordinal pattern positioned slopes (COPPS). We also constructed horizontal visibility graphs and estimated the exponent of the decay of the degree distribution. Taken together, the analyses create a characterization of the dynamics of these systems which can be scrutinized for universality, either across variables or between the three geographically very close catchments.","PeriodicalId":11694,"journal":{"name":"Entropy","volume":"27 4","pages":""},"PeriodicalIF":2.1,"publicationDate":"2025-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12025984/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144003979","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Exploring ISAC: Information-Theoretic Insights. 探索ISAC：信息论见解。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2025-04-02 DOI: 10.3390/e27040378

Mehrasa Ahmadipour, Michèle Wigger, Shlomo Shamai

{"title":"Exploring ISAC: Information-Theoretic Insights.","authors":"Mehrasa Ahmadipour, Michèle Wigger, Shlomo Shamai","doi":"10.3390/e27040378","DOIUrl":"https://doi.org/10.3390/e27040378","url":null,"abstract":"This article reviews results from the literature illustrating the bottlenecks and tradeoffs of integrated sensing and communication (ISAC) through the lens of information theory, thus offering a distinct perspective compared to recent works that focus on signal processing, wireless communications, or other related overviews. Different models and scenarios are considered and compared. For example, scenarios where radar sensing is performed at the communication and radar transmitter (mono-static ISAC) and scenarios where the radar receiver differs from the radar transmitter (called bi-static radar). Similarly, we discuss ISAC bottlenecks and tradeoffs both in slowly-varying environments where the main sensing target is described by a single parameter and accordingly, sensing performance is described by detection error probabilities, as well as in fast-varying environments, where the sensing targets are described by vectors and thus vector-valued performance measures such as average distortions like mean-squared errors are used to determine sensing performances. This overview article further also considers limitations and opportunities in network ISAC environments, such as collaborative or interactive sensing, and the influence of secrecy and privacy requirements on ISAC systems, a line of research that has received growing interest over the last few years. For all these scenarios, we provide and discuss precise models and their limitations and provide either bounds or full characterizations of the fundamental information-theoretic performance limits of these systems. Further extensions as well as important open research directions are also discussed.","PeriodicalId":11694,"journal":{"name":"Entropy","volume":"27 4","pages":""},"PeriodicalIF":2.1,"publicationDate":"2025-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12026050/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143964230","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Lightweight Pre-Trained Korean Language Model Based on Knowledge Distillation and Low-Rank Factorization. 基于知识蒸馏和低秩分解的轻量级预训练韩语模型。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2025-04-02 DOI: 10.3390/e27040379

Jin-Hwan Kim, Young-Seok Choi

{"title":"Lightweight Pre-Trained Korean Language Model Based on Knowledge Distillation and Low-Rank Factorization.","authors":"Jin-Hwan Kim, Young-Seok Choi","doi":"10.3390/e27040379","DOIUrl":"https://doi.org/10.3390/e27040379","url":null,"abstract":"Natural Language Processing (NLP) stands as a forefront of artificial intelligence research, empowering computational systems to comprehend and process human language as used in everyday contexts. Language models (LMs) underpin this field, striving to capture the intricacies of linguistic structure and semantics by assigning probabilities to sequences of words. The trend towards large language models (LLMs) has shown significant performance improvements with increasing model size. However, the deployment of LLMs on resource-limited devices such as mobile and edge devices remains a challenge. This issue is particularly pronounced in languages other than English, including Korean, where pre-trained models are relatively scarce. Addressing this gap, we introduce a novel lightweight pre-trained Korean language model that leverages knowledge distillation and low-rank factorization techniques. Our approach distills knowledge from a 432 MB (approximately 110 M parameters) teacher model into student models of substantially reduced sizes (e.g., 53 MB ≈ 14 M parameters, 35 MB ≈ 13 M parameters, 30 MB ≈ 11 M parameters, and 18 MB ≈ 4 M parameters). The smaller student models further employ low-rank factorization to minimize the parameter count within the Transformer's feed-forward network (FFN) and embedding layer. We evaluate the efficacy of our lightweight model across six established Korean NLP tasks. Notably, our most compact model, KR-ELECTRA-Small-KD, attains over 97.387% of the teacher model's performance despite an 8.15× reduction in size. Remarkably, on the NSMC sentiment classification benchmark, KR-ELECTRA-Small-KD surpasses the teacher model with an accuracy of 89.720%. These findings underscore the potential of our model as an efficient solution for NLP applications in resource-constrained settings.","PeriodicalId":11694,"journal":{"name":"Entropy","volume":"27 4","pages":""},"PeriodicalIF":2.1,"publicationDate":"2025-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12026428/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143963169","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Bit-Level Construction for Multiplicative-Repetition-Based Non-Binary Polar Codes. 基于乘法重复的非二进制极码的位级构造。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2025-04-02 DOI: 10.3390/e27040377

Rongchi Xu, Peiyao Chen, Ling Liu, Min Zhu, Baoming Bai

{"title":"Bit-Level Construction for Multiplicative-Repetition-Based Non-Binary Polar Codes.","authors":"Rongchi Xu, Peiyao Chen, Ling Liu, Min Zhu, Baoming Bai","doi":"10.3390/e27040377","DOIUrl":"https://doi.org/10.3390/e27040377","url":null,"abstract":"In this paper, we discuss non-binary polar codes using a 2×2 matrix over a Galois field GF(2q) as the kernel. Conventional construction of non-binary polar codes divides the synthesized channels into frozen channels and information channels. Each information channel carries one symbol, i.e., q bits. However, there are many middle channels with insufficient polarization, which cannot carry one symbol of q bits but only i bits, 1≤i<q,i∈Z, at finite block length. In this paper, we consider bit-level construction for multiplicative repetition (MR)-based non-binary polar codes and propose a bit-level construction based on the two following methods. We first calculate the error probability and channel capacity lower bound of each synthesized channel based on the channel degradation method, and then determine both the number and index of the carried bits for each synthesized channel according to the symbol error probability and capacity. To reduce complexity, we also introduce a Monte-Carlo method. We compute the error probability of each synthesized channel carrying i information bits and select the optimal construction that can minimize the union bound of the error probability. Finally, an improved construction-based probabilistic shaping method for MR-based non-binary polar codes is considered. Simulation results show that the proposed construction significantly improved the decoding performance compared with the conventional construction scheme.","PeriodicalId":11694,"journal":{"name":"Entropy","volume":"27 4","pages":""},"PeriodicalIF":2.1,"publicationDate":"2025-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12026330/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143987685","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Family of Optimal Linear Functional-Repair Regenerating Storage Codes. 一类最优线性功能修复再生存储代码。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2025-04-01 DOI: 10.3390/e27040376

Henk D L Hollmann

{"title":"A Family of Optimal Linear Functional-Repair Regenerating Storage Codes.","authors":"Henk D L Hollmann","doi":"10.3390/e27040376","DOIUrl":"https://doi.org/10.3390/e27040376","url":null,"abstract":"We construct a family of linear optimal functional-repair regenerating storage codes with parameters {m,(n,k),(r,α,β)}={(2r-α+1)α/2,(r+1,r),(r,α,1)} for any integers r,α with 1≤α≤r, over any field when α∈{1,r-1,r}, and over any finite field Fq with q≥r-1 otherwise. These storage codes are Minimum-Storage Regenerating (MSR) when α=1, Minimum-Bandwidth Regenerating (MBR) when α=r, and represents extremal points of the (convex) attainable cut-set region different from the MSR and MBR points in all other cases. It is known that when 2≤α≤r-1, these parameters cannot be realized by exact-repair storage codes. Each of these codes come with an explicit and relatively simple repair method, and repair can even be realized as help-by-transfer (HBT) if desired. The coding states of codes from this family can be described geometrically as configurations of r+1 subspaces of dimension α in an m-dimensional vector space with restricted sub-span dimensions. A few \"small\" codes with these parameters are known: one for (r,α)=(3,2) dating from 2013 and one for (r,α)=(4,3) dating from 2024. Apart from these, our codes are the first examples of explicit, relatively simple, optimal functional-repair storage codes over a small finite field, with an explicit repair method and with parameters representing an extremal point of the attainable cut-set region distinct from the MSR and MBR points.","PeriodicalId":11694,"journal":{"name":"Entropy","volume":"27 4","pages":""},"PeriodicalIF":2.1,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12025674/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143959363","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Feature-Driven Semantic Communication for Efficient Image Transmission. 面向高效图像传输的特征驱动语义通信。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2025-03-31 DOI: 10.3390/e27040369

Ji Zhang, Ying Zhang, Baofeng Ji, Anmin Chen, Aoxue Liu, Hengzhou Xu

{"title":"Feature-Driven Semantic Communication for Efficient Image Transmission.","authors":"Ji Zhang, Ying Zhang, Baofeng Ji, Anmin Chen, Aoxue Liu, Hengzhou Xu","doi":"10.3390/e27040369","DOIUrl":"https://doi.org/10.3390/e27040369","url":null,"abstract":"Semantic communication is an emerging approach that enhances transmission efficiency by conveying the semantic content of information more effectively. It has garnered significant attention in recent years. However, existing semantic communication systems for image transmission typically adopt direct transmission of features or uniformly compress features before transmission. They have not yet considered the differential impact of features on image recovery at the receiver end and the issue of bandwidth limitations during actual transmission. This paper shows that non-uniform processing of features leads to better image recovery under bandwidth constraints compared to uniform processing. Based on this, we propose a semantic communication system for image transmission, which introduces non-uniform quantization techniques. In the feature transmission stage, the system performs varying levels of quantization based on the differences in feature performance at the receiver, thereby reducing the bandwidth requirement. Inspired by quantitative quantization techniques, we design a non-uniform quantization algorithm capable of dynamic bit allocation. This algorithm, under bandwidth constraints, dynamically adjusts the quantization precision of features based on their contribution to the completion of tasks at the receiver end, ensuring the quality and accuracy of the transmitted data even under limited bandwidth conditions. Experimental results show that the proposed system reduces bandwidth usage while ensuring image reconstruction quality.","PeriodicalId":11694,"journal":{"name":"Entropy","volume":"27 4","pages":""},"PeriodicalIF":2.1,"publicationDate":"2025-03-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12025675/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143964346","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Correlations and Kappa Distributions: Numerical Experiment and Physical Understanding. 相关性和Kappa分布：数值实验和物理理解。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2025-03-31 DOI: 10.3390/e27040375

David J McComas, George Livadiotis, Nicholas V Sarlis

{"title":"Correlations and Kappa Distributions: Numerical Experiment and Physical Understanding.","authors":"David J McComas, George Livadiotis, Nicholas V Sarlis","doi":"10.3390/e27040375","DOIUrl":"https://doi.org/10.3390/e27040375","url":null,"abstract":"Kappa distributions, their statistical framework, and their thermodynamic origin describe systems with correlations among their particle energies, residing in stationary states out of classical thermal equilibrium/space plasmas, from solar wind to the outer heliosphere, are such systems. We show how correlations from long-range interactions compete with collisions to define the specific shape of particle velocity distributions, using a simple numerical experiment with collisions and a variable amount of correlation among the particles. When the correlations are turned off, collisions drive any initial distribution to evolve toward equilibrium and a Maxwell-Boltzmann (MB) distribution. However, when some correlation is introduced, the distribution evolves toward a different stationary state defined by a kappa distribution with some finite value of the thermodynamic kappa κ (where κ→∞ corresponds to a MB distribution). Furthermore, the stronger the correlations, the lower the κ value. This simple numerical experiment illuminates the role of correlations in forming stationary state particle distributions, which are described by kappa distributions, as well as the physical interpretation of correlations from long-range interactions and how they are related to the thermodynamic kappa.","PeriodicalId":11694,"journal":{"name":"Entropy","volume":"27 4","pages":""},"PeriodicalIF":2.1,"publicationDate":"2025-03-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12025533/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143998197","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Successive Refinement for Lossy Compression of Individual Sequences. 单个序列有损压缩的逐次细化。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2025-03-31 DOI: 10.3390/e27040370

Neri Merhav

引用次数: 0

Latent Abstractions in Generative Diffusion Models. 生成扩散模型中的潜在抽象。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2025-03-31 DOI: 10.3390/e27040371

Giulio Franzese, Mattia Martini, Giulio Corallo, Paolo Papotti, Pietro Michiardi

引用次数: 0

Entropy最新文献