Entropy最新文献 - Book学术

Inferring About the Average Value of Audit Errors from Sequential Ratio Tests. 从连续比率测试推断审计错误的平均值。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2024-11-20 DOI: 10.3390/e26110998

Grzegorz Sitek, Mariusz Pleszczyński

引用次数: 0

How Do Transformers Model Physics? Investigating the Simple Harmonic Oscillator. 变压器如何模拟物理？研究简谐振荡器

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2024-11-19 DOI: 10.3390/e26110997

Subhash Kantamneni, Ziming Liu, Max Tegmark

{"title":"How Do Transformers Model Physics? Investigating the Simple Harmonic Oscillator.","authors":"Subhash Kantamneni, Ziming Liu, Max Tegmark","doi":"10.3390/e26110997","DOIUrl":"10.3390/e26110997","url":null,"abstract":"How do transformers model physics? Do transformers model systems with interpretable analytical solutions or do they create an \"alien physics\" that is difficult for humans to decipher? We have taken a step towards demystifying this larger puzzle by investigating the simple harmonic oscillator (SHO), x¨+2γx˙+ω02x=0, one of the most fundamental systems in physics. Our goal was to identify the methods transformers use to model the SHO, and to do so we hypothesized and evaluated possible methods by analyzing the encoding of these methods' intermediates. We developed four criteria for the use of a method within the simple test bed of linear regression, where our method was y=wx and our intermediate was w: (1) Can the intermediate be predicted from hidden states? (2) Is the intermediate's encoding quality correlated with the model performance? (3) Can the majority of variance in hidden states be explained by the intermediate? (4) Can we intervene on hidden states to produce predictable outcomes? Armed with these two correlational (1,2), weak causal (3), and strong causal (4) criteria, we determined that transformers use known numerical methods to model the trajectories of the simple harmonic oscillator, specifically, the matrix exponential method. Our analysis framework can conveniently extend to high-dimensional linear systems and nonlinear systems, which we hope will help reveal the \"world model\" hidden in transformers.","PeriodicalId":11694,"journal":{"name":"Entropy","volume":"26 11","pages":""},"PeriodicalIF":2.1,"publicationDate":"2024-11-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11592621/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142727218","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

"In Mathematical Language": On Mathematical Foundations of Quantum Foundations. "用数学语言"：论量子基础的数学基础》。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2024-11-18 DOI: 10.3390/e26110989

Arkady Plotnitsky

{"title":"\"In Mathematical Language\": On Mathematical Foundations of Quantum Foundations.","authors":"Arkady Plotnitsky","doi":"10.3390/e26110989","DOIUrl":"10.3390/e26110989","url":null,"abstract":"The argument of this article is threefold. First, the article argues that from its rise in the sixteenth century to our own time, the advancement of modern physics as mathematical-experimental science has been defined by the invention of new mathematical structures. Second, the article argues that quantum theory, especially following quantum mechanics, gives this thesis a radically new meaning by virtue of the following two features: on the one hand, quantum phenomena are defined as essentially different from those found in all previous physics by purely physical features; and on the other, quantum mechanics and quantum field theory are defined by purely mathematical postulates, which connect them to quantum phenomena strictly in terms of probabilities, without, as in all previous physics, representing or otherwise relating to how these phenomena physically come about. While these two features may appear discordant, if not inconsistent, I argue that they are in accord with each other, at least in certain interpretations (including the one adopted here), designated as \"reality without realism\", RWR, interpretations. This argument also allows this article to offer a new perspective on a thorny problem of the relationships between continuity and discontinuity in quantum physics. In particular, rather than being concerned only with the discreteness and continuity of quantum objects or phenomena, quantum mechanics and quantum field theory relate their continuous mathematics to the irreducibly discrete quantum phenomena in terms of probabilistic predictions while, at least in RWR interpretations, precluding a representation or even conception of how these phenomena come about. This subject is rarely, if ever, discussed apart from previous work by the present author. It is, however, given a new dimension in this article which introduces, as one of its main contributions, a new principle: the mathematical complexity principle.","PeriodicalId":11694,"journal":{"name":"Entropy","volume":"26 11","pages":""},"PeriodicalIF":2.1,"publicationDate":"2024-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11593060/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142726637","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Derangetropy in Probability Distributions and Information Dynamics. 概率分布和信息动态中的脱熵。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2024-11-18 DOI: 10.3390/e26110996

Masoud Ataei, Xiaogang Wang

引用次数: 0

Integration of Sense and Control for Uncertain Systems Based on Delayed Feedback Active Inference. 基于延迟反馈主动推理的不确定系统的感知与控制集成。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2024-11-18 DOI: 10.3390/e26110990

Mingyue Ji, Kunpeng Pan, Xiaoxuan Zhang, Quan Pan, Xiangcheng Dai, Yang Lyu

{"title":"Integration of Sense and Control for Uncertain Systems Based on Delayed Feedback Active Inference.","authors":"Mingyue Ji, Kunpeng Pan, Xiaoxuan Zhang, Quan Pan, Xiangcheng Dai, Yang Lyu","doi":"10.3390/e26110990","DOIUrl":"10.3390/e26110990","url":null,"abstract":"Asa result of the time lag in transmission, the data obtained by the sensor is delayed and does not reflect the state at the current moment. The effects of input delay are often overlooked in active inference (AIF), which may lead to significant deviations in state estimation and increased prediction errors, particularly when the system is subjected to a sudden external stimulus. In this paper, a theoretical framework of delayed feedback active inference (DAIF) is proposed to enhance the applicability of AIF to real systems. The probability model of DAIF is defined by incorporating a control distribution into that of AIF. The free energy of DAIF is defined as the sum of the quadratic state, sense, and control prediction error. A predicted state derived from previous states is defined and introduced as the expectation of the prior distribution of the real-time state. A proportional-integral (PI)-like control based on the predicted state is taken to be the expectation of DAIF preference control, whose gain coefficient is inversely proportional to the measurement accuracy variance. To adaptively compensate for external disturbances, a second-order inverse variance accuracy replaces the fixed sensory accuracy of preference control. The simulation results of the trajectory tracking control of a quadrotor unmanned aerial vehicle (UAV) show that DAIF performs better than AIF in state estimation and disturbance resistance.","PeriodicalId":11694,"journal":{"name":"Entropy","volume":"26 11","pages":""},"PeriodicalIF":2.1,"publicationDate":"2024-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11592534/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142727231","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Mathematical Modeling of Physical Reality: From Numbers to Fractals, Quantum Mechanics and the Standard Model. 物理现实的数学建模：从数字到分形》、《量子力学与标准模型》。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2024-11-18 DOI: 10.3390/e26110991

Marian Kupczynski

{"title":"Mathematical Modeling of Physical Reality: From Numbers to Fractals, Quantum Mechanics and the Standard Model.","authors":"Marian Kupczynski","doi":"10.3390/e26110991","DOIUrl":"10.3390/e26110991","url":null,"abstract":"In physics, we construct idealized mathematical models in order to explain various phenomena which we observe or create in our laboratories. In this article, I recall how sophisticated mathematical models evolved from the concept of a number created thousands of years ago, and I discuss some challenges and open questions in quantum foundations and in the Standard Model. We liberated nuclear energy, landed on the Moon and built 'quantum computers'. Encouraged by these successes, many believe that when we reconcile general relativity with quantum theory we will have the correct theory of everything. Perhaps we should be much humbler. Our perceptions of reality are biased by our senses and by our brain, bending them to meet our priors and expectations. Our abstract mathematical models describe only in an approximate way different layers of physical reality. To describe the motion of a meteorite, we can use a concept of a material point, but the point-like approximation breaks completely when the meteorite hits the Earth. Similarly, thermodynamic, chemical, molecular, atomic, nuclear and elementary particle layers of physical reality are described using specific abstract mathematical models and approximations. In my opinion, the theory of everything does not exist.","PeriodicalId":11694,"journal":{"name":"Entropy","volume":"26 11","pages":""},"PeriodicalIF":2.1,"publicationDate":"2024-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11592783/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142727288","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Information-Theoretic Generalization Bounds for Batch Reinforcement Learning. 批量强化学习的信息论泛化界限

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2024-11-18 DOI: 10.3390/e26110995

Xingtu Liu

引用次数: 0

Generalized Filter Bank Orthogonal Frequency Division Multiplexing: Low-Complexity Waveform for Ultra-Wide Bandwidth and Flexible Services. 通用滤波器组正交频分复用：用于超宽带宽和灵活服务的低复杂度波形。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2024-11-18 DOI: 10.3390/e26110994

Yu Xin, Jian Hua, Tong Bao, Yaxing Hao, Ziheng Xiao, Xin Nie, Fanggang Wang

{"title":"Generalized Filter Bank Orthogonal Frequency Division Multiplexing: Low-Complexity Waveform for Ultra-Wide Bandwidth and Flexible Services.","authors":"Yu Xin, Jian Hua, Tong Bao, Yaxing Hao, Ziheng Xiao, Xin Nie, Fanggang Wang","doi":"10.3390/e26110994","DOIUrl":"10.3390/e26110994","url":null,"abstract":"Terahertz (THz) communication is a crucial technique in sixth generation (6G) mobile networks, which allow for ultra-wide bandwidths to enable ultra-high data rate wireless communication. However, the current subcarrier spacing and the size of fast Fourier transform (FFT) of the orthogonal frequency division multiplexing (OFDM) in 5G NR are insufficient regarding the bandwidth requirements of terahertz scenarios. In this paper, a novel waveform is proposed to address the ultra-wideband issue, namely the generalized filter bank orthogonal frequency division multiplexing (GFB-OFDM) waveform. The main advantages are summarized as follows: (1) The K-point IFFT is implemented by two levels of IFFTs in smaller sizes, i.e, performing M-point IFFT in N times and performing N-point IFFT in M times, where K=N×M. (2) The proposed waveform can accommodate flexible subcarrier spacings and different numbers of the subbands to provide various services in a single GFB-OFDM symbol. (3) Different bandwidths can be supported using a fixed filter since the filtering is performed on each subband. In contrast, the cyclic prefix orthogonal frequency division multiplexing (CP-OFDM) in 4G/5G requires various filters. (4) The existing detection for CP-OFDM can be directly employed as the detector of the proposed waveform. Lastly, the comprehensive simulation results demonstrate that GFB-OFDM outperforms CP-OFDM in terms of the out-of-band leakage, complexity and error performance.","PeriodicalId":11694,"journal":{"name":"Entropy","volume":"26 11","pages":""},"PeriodicalIF":2.1,"publicationDate":"2024-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11592793/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142727205","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multi-Label Feature Selection with Feature-Label Subgraph Association and Graph Representation Learning. 利用特征-标签子图关联和图表示学习进行多标签特征选择

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2024-11-18 DOI: 10.3390/e26110992

Jinghou Ruan, Mingwei Wang, Deqing Liu, Maolin Chen, Xianjun Gao

{"title":"Multi-Label Feature Selection with Feature-Label Subgraph Association and Graph Representation Learning.","authors":"Jinghou Ruan, Mingwei Wang, Deqing Liu, Maolin Chen, Xianjun Gao","doi":"10.3390/e26110992","DOIUrl":"10.3390/e26110992","url":null,"abstract":"In multi-label data, a sample is associated with multiple labels at the same time, and the computational complexity is manifested in the high-dimensional feature space as well as the interdependence and unbalanced distribution of labels, which leads to challenges regarding feature selection. As a result, a multi-label feature selection method based on feature-label subgraph association with graph representation learning (SAGRL) is proposed to represent the complex correlations of features and labels, especially the relationships between features and labels. Specifically, features and labels are mapped to nodes in the graph structure, and the connections between nodes are established to form feature and label sets, respectively, which increase intra-class correlation and decrease inter-class correlation. Further, feature-label subgraphs are constructed by feature and label sets to provide abundant feature combinations. The relationship between each subgraph is adjusted by graph representation learning, the crucial features in different label sets are selected, and the optimal feature subset is obtained by ranking. Experimental studies on 11 datasets show the superior performance of the proposed method with six evaluation metrics over some state-of-the-art multi-label feature selection methods.","PeriodicalId":11694,"journal":{"name":"Entropy","volume":"26 11","pages":""},"PeriodicalIF":2.1,"publicationDate":"2024-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11592953/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142727290","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Still No Evidence for an Effect of the Proportion of Non-Native Speakers on Natural Language Complexity. 仍无证据表明非母语者比例对自然语言复杂性有影响。

IF 2.1 3区物理与天体物理

Entropy Pub Date : 2024-11-18 DOI: 10.3390/e26110993

Alexander Koplenig

{"title":"Still No Evidence for an Effect of the Proportion of Non-Native Speakers on Natural Language Complexity.","authors":"Alexander Koplenig","doi":"10.3390/e26110993","DOIUrl":"10.3390/e26110993","url":null,"abstract":"In a recent study, I demonstrated that large numbers of L2 (second language) speakers do not appear to influence the morphological or information-theoretic complexity of natural languages. This paper has three primary aims: First, I address recent criticisms of my analyses, showing that the points raised by my critics were already explicitly considered and analysed in my original work. Furthermore, I show that the proposed alternative analyses fail to withstand detailed examination. Second, I introduce new data on the information-theoretic complexity of natural languages, with the estimates derived from various language models-ranging from simple statistical models to advanced neural networks-based on a database of 40 multilingual text collections that represent a wide range of text types. Third, I re-analyse the information-theoretic and morphological complexity data using novel methods that better account for model uncertainty in parameter estimation, as well as the genealogical relatedness and geographic proximity of languages. In line with my earlier findings, the results show no evidence that large numbers of L2 speakers have an effect on natural language complexity.","PeriodicalId":11694,"journal":{"name":"Entropy","volume":"26 11","pages":""},"PeriodicalIF":2.1,"publicationDate":"2024-11-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11593100/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142727291","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0