Physica A: Statistical Mechanics and its Applications最新文献

筛选
英文 中文
Dynamic probabilistic decision networks 动态概率决策网络
IF 3.1 3区 物理与天体物理
Physica A: Statistical Mechanics and its Applications Pub Date : 2026-04-01 Epub Date: 2026-02-11 DOI: 10.1016/j.physa.2026.131382
V.I. Yukalov , E.P. Yukalova
{"title":"Dynamic probabilistic decision networks","authors":"V.I. Yukalov ,&nbsp;E.P. Yukalova","doi":"10.1016/j.physa.2026.131382","DOIUrl":"10.1016/j.physa.2026.131382","url":null,"abstract":"<div><div>A new type of decision networks is suggested and its operation is analyzed. The network nodes are represented by intelligent agents who can denote either some biological beings, like humans, or neurons of the brain, or the nodes of artificial intelligence. The specifics of the network are in the following: It is <em>probabilistic</em> in the sense that the choice, accomplished by each agent, is characterized by the related probability. It is <em>dynamic</em>, with the probabilities varying in time due to the exchange of information between the agents. It is <em>affective</em>, because the agents choose between alternatives by taking account of utility as well as of biases and emotions. In general, it is <em>heterogeneous</em>, being composed of the groups of agents with different properties, for instance having long-term memory and short-term memory. The network dynamics, caused by the information exchange, results in decision error decrease. The network operation is illustrated by the example starting with the Allais paradox, its resolution, and the decision error diminution in the process of decision dynamics with information exchange. Resorting to machine-learning techniques it is possible to regulate the behavior of the network agents forcing them to choose particular alternatives.</div></div>","PeriodicalId":20152,"journal":{"name":"Physica A: Statistical Mechanics and its Applications","volume":"687 ","pages":"Article 131382"},"PeriodicalIF":3.1,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146192943","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Invariant-Net: Enhancing generalization in physics-informed neural networks via lie symmetry reduction 不变性网络:通过谎言对称约简增强物理信息神经网络的泛化
IF 3.1 3区 物理与天体物理
Physica A: Statistical Mechanics and its Applications Pub Date : 2026-04-01 Epub Date: 2026-02-12 DOI: 10.1016/j.physa.2026.131383
Pengcheng Cheng , Shuhuan Li
{"title":"Invariant-Net: Enhancing generalization in physics-informed neural networks via lie symmetry reduction","authors":"Pengcheng Cheng ,&nbsp;Shuhuan Li","doi":"10.1016/j.physa.2026.131383","DOIUrl":"10.1016/j.physa.2026.131383","url":null,"abstract":"<div><div>Physics-Informed Neural Networks (PINNs) have emerged as a powerful tool for solving partial differential equations (PDEs). However, they fundamentally function as high-dimensional interpolators, often failing to capture the underlying causal dynamics required for out-of-distribution (OOD) extrapolation. In this work, we propose Invariant-Net, a geometrically grounded architecture that embeds Lie point symmetries directly into the learning process. Instead of approximating the solution in the original spatiotemporal domain, Invariant-Net projects the inputs onto a lower-dimensional quotient manifold determined by the symmetry group of the PDE. This effectively reduces the learning task from solving a high-dimensional PDE to approximating a simpler, often lower-dimensional, invariant function (e.g., a self-similar profile). We validate our framework on a diverse set of systems including the Porous Medium Equation (scaling symmetry), the Korteweg–de Vries equation (Galilean invariance), the 2D Heat equation (rotational symmetry), and the Nonlinear Schrödinger equation (internal phase symmetry). Our experiments demonstrate two critical advantages: (1) Dimensionality Reduction, which breaks the curse of dimensionality and significantly improves sample efficiency; and (2) Robust Generalization, where standard PINNs fail catastrophically. Specifically, in the time-extrapolation regime of the Porous Medium Equation, standard PINNs exhibit an error explosion from 28% to 75%, whereas Invariant-Net maintains a stable error of <span><math><mrow><mo>≈</mo><mn>5</mn><mtext>%</mtext></mrow></math></span> indefinitely into the future. These results confirm that enforcing exact geometric symmetries allows neural networks to learn the intrinsic physical laws rather than merely fitting observed data.</div></div>","PeriodicalId":20152,"journal":{"name":"Physica A: Statistical Mechanics and its Applications","volume":"687 ","pages":"Article 131383"},"PeriodicalIF":3.1,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146192942","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
An adaptive multi-task learning method with task-specific attention for node classification and link prediction 一种针对特定任务的自适应多任务学习方法,用于节点分类和链路预测
IF 3.1 3区 物理与天体物理
Physica A: Statistical Mechanics and its Applications Pub Date : 2026-04-01 Epub Date: 2026-01-30 DOI: 10.1016/j.physa.2026.131343
Mingzhou Yang , Yuxin Liu
{"title":"An adaptive multi-task learning method with task-specific attention for node classification and link prediction","authors":"Mingzhou Yang ,&nbsp;Yuxin Liu","doi":"10.1016/j.physa.2026.131343","DOIUrl":"10.1016/j.physa.2026.131343","url":null,"abstract":"<div><div>Multi-task learning has shown great potential for joint modeling of node classification and link prediction on graph-structured data. However, many existing multi-task GNNs still rely on rigid shared representations with limited adaptive fusion capability, making it difficult to effectively integrate global aggregation and local attention and to properly disentangle task-specific representations, which in turn leads to negative transfer. In addition, task imbalance during optimization can cause one objective to dominate training, resulting in unstable convergence and degraded performance. To address these challenges, we propose an adaptive multi-task learning framework for node classification and link prediction, called AMTGG, which adaptively coordinates both tasks within a unified architecture. Specifically, AMTGG constructs shared representations via parallel GCN and GAT branches, and introduces learnable fusion coefficients to adaptively combine their complementary global–local cues. Then, task-specific attention layers and gating mechanisms are employed to automatically select task-relevant subspaces from the shared features, thereby alleviating representation interference between objectives. To balance the training dynamics of the two tasks, GradNorm is adopted to dynamically adjust task weights, while EMA smoothing is further applied to stabilize weight evolution and improve convergence. Since link prediction involves modeling the interaction between node pairs, which is often underemphasized in multi-task frameworks, our model incorporates subgraph extraction and line graph transformation to enhance edge-level structural learning. Extensive experiments on seven benchmark graph datasets demonstrate that AMTGG enhances model robustness through feature masking regularization and noise injection, and achieves synergistic improvements on both node classification and link prediction.</div></div>","PeriodicalId":20152,"journal":{"name":"Physica A: Statistical Mechanics and its Applications","volume":"687 ","pages":"Article 131343"},"PeriodicalIF":3.1,"publicationDate":"2026-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146147484","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Comment on “Parity–breaking Ising–like model with Spin-Exchange dynamics” [Physica A 677 (2025) 130938] 关于“具有自旋交换动力学的类奇偶破缺伊辛模型”的评析[物理学报A 677 (2025) 130938]
IF 3.1 3区 物理与天体物理
Physica A: Statistical Mechanics and its Applications Pub Date : 2026-03-31 DOI: 10.1016/j.physa.2026.131536
Matteo Morello , Alessandro Pelizzola , Marco Pretti
{"title":"Comment on “Parity–breaking Ising–like model with Spin-Exchange dynamics” [Physica A 677 (2025) 130938]","authors":"Matteo Morello ,&nbsp;Alessandro Pelizzola ,&nbsp;Marco Pretti","doi":"10.1016/j.physa.2026.131536","DOIUrl":"10.1016/j.physa.2026.131536","url":null,"abstract":"<div><div>In this Comment, we discuss some of the conclusions that have been drawn in the paper “Parity–breaking Ising–like model with Spin–Exchange dynamics” [Physica A 677 (2025) 130938], where a one–dimensional Ising–like model with a parity–breaking interaction and a spin–exchange (Kawasaki) dynamics has been investigated by means of Wang–Landau Monte Carlo simulations. We use both Wang–Landau Monte Carlo simulations and, for small system size, exact results. Firstly, we address the issue of the existence of a finite–temperature phase transition, showing that this is not supported by results for the specific heat. Then, we show that the ground state energy at low density is slightly smaller than the value reported in the commented paper. Finally, we address the issue of the existence of a non–vanishing equilibrium current, and we (i) show that in the pure Ising case the equilibrium current vanishes and (ii) argue that in presence of the parity–breaking interaction the reason why this current is non–zero is that the model is simulated using the Wang–Landau algorithm or, more generally, an algorithm whose stationary state is not in detailed balance with the Boltzmann equilibrium distribution.</div></div>","PeriodicalId":20152,"journal":{"name":"Physica A: Statistical Mechanics and its Applications","volume":"692 ","pages":"Article 131536"},"PeriodicalIF":3.1,"publicationDate":"2026-03-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147614001","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Pedestrian evacuation in fire environments: Scientometric analysis and systematic evaluation 火灾环境下的行人疏散:科学计量分析和系统评价
IF 3.1 3区 物理与天体物理
Physica A: Statistical Mechanics and its Applications Pub Date : 2026-03-15 Epub Date: 2026-01-25 DOI: 10.1016/j.physa.2026.131323
Xinyue Qi , Xiaolian Li , Jun Zhang , Weiguo Song
{"title":"Pedestrian evacuation in fire environments: Scientometric analysis and systematic evaluation","authors":"Xinyue Qi ,&nbsp;Xiaolian Li ,&nbsp;Jun Zhang ,&nbsp;Weiguo Song","doi":"10.1016/j.physa.2026.131323","DOIUrl":"10.1016/j.physa.2026.131323","url":null,"abstract":"<div><div>Understanding pedestrian evacuation behaviors in fire is critical to mitigate human casualties in emergencies. Benefiting from the rapid progress in fire safety engineering and computer simulation technologies, research on pedestrian evacuation in fire environments has attracted increasing attention from the academic community. To have a comprehensive understanding on the hotspots and development trends in this field, in this study a total of 673 relevant literatures from 2000 to 2024 have been collected from the database Web of Science and Scopus. The keywords co-occurrence network, authors' collaborative network, journal productivity, regional cooperation and research themes are analyzed. The results show that the amount of literature related to fire evacuation exhibit an exponential increase on the whole since 2015. The most productive journal is Physica A-statistical Mechanics and Its Applications, with 76 published articles, accounting for 11.3 % of the total research publications. The topic analysis shows that the research hotspots focus on the movement behavior and characteristics of the crowd in the fire environment, the simulation and optimization of the evacuation process, and the individual's risk perception and decision-making behavior. Further, the study identifies the key challenges for improving pedestrian emergency evacuation capabilities and suggests possible research directions.</div></div>","PeriodicalId":20152,"journal":{"name":"Physica A: Statistical Mechanics and its Applications","volume":"686 ","pages":"Article 131323"},"PeriodicalIF":3.1,"publicationDate":"2026-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146081893","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Impact of uncertain payoffs on evolution of cooperation in TFT-AllD game based on imitation dynamics 基于模仿动力学的TFT-AllD博弈中不确定收益对合作演化的影响
IF 3.1 3区 物理与天体物理
Physica A: Statistical Mechanics and its Applications Pub Date : 2026-03-15 Epub Date: 2026-02-02 DOI: 10.1016/j.physa.2026.131351
Si-Yi Wang , Zi-Xuan Li , Xiu-Deng Zheng , Yi Tao , Cong Li
{"title":"Impact of uncertain payoffs on evolution of cooperation in TFT-AllD game based on imitation dynamics","authors":"Si-Yi Wang ,&nbsp;Zi-Xuan Li ,&nbsp;Xiu-Deng Zheng ,&nbsp;Yi Tao ,&nbsp;Cong Li","doi":"10.1016/j.physa.2026.131351","DOIUrl":"10.1016/j.physa.2026.131351","url":null,"abstract":"<div><div>In order to explore how the evolution of cooperation in TFT-AllD games based on imitation dynamics is affected by the uncertainty of payoffs (or environmental noise), the stochastic evolutionary and convergence stabilities of this system are investigated. We here assume that the uncertainty in payoffs arises from the stochasticity in the number of repeated interactions among individuals. The results show that: (<em>i</em>) the stochastic evolutionary stability is only determined by the properties of the average payoff matrix; (<em>ii</em>) not only the pure strategies <span><math><mrow><mi>T</mi><mi>F</mi><mi>T</mi></mrow></math></span> (Tit-for-Tat) and <span><math><mrow><mi>A</mi><mi>l</mi><mi>l</mi><mi>D</mi></mrow></math></span> (Always Defect) could be both stochastically evolutionarily stable (SES), but also a completely mixed strategy corresponding to the interior equilibrium of the average payoff matrix must be also SES; and (<em>iii</em>) a completely mixed strategy must be not stochastically convergence stable (SCS) even though it is SES. In addition, we show also that for the TFT-AllD games based on imitation dynamics in finite populations, the weak selection can filter the impact of uncertain payoffs on the fixation probability of <span><math><mrow><mi>T</mi><mi>F</mi><mi>T</mi></mrow></math></span>.</div></div>","PeriodicalId":20152,"journal":{"name":"Physica A: Statistical Mechanics and its Applications","volume":"686 ","pages":"Article 131351"},"PeriodicalIF":3.1,"publicationDate":"2026-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147385718","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A similarity laplacian energy perspective on critical node identification in complex networks 复杂网络中关键节点识别的相似拉普拉斯能量视角
IF 3.1 3区 物理与天体物理
Physica A: Statistical Mechanics and its Applications Pub Date : 2026-03-15 Epub Date: 2026-02-02 DOI: 10.1016/j.physa.2026.131349
Sizheng Liu , Xinghua Qin , Yirun Ruan , Mengmeng Zhang , Liang Bai , Tianyuan Yu
{"title":"A similarity laplacian energy perspective on critical node identification in complex networks","authors":"Sizheng Liu ,&nbsp;Xinghua Qin ,&nbsp;Yirun Ruan ,&nbsp;Mengmeng Zhang ,&nbsp;Liang Bai ,&nbsp;Tianyuan Yu","doi":"10.1016/j.physa.2026.131349","DOIUrl":"10.1016/j.physa.2026.131349","url":null,"abstract":"<div><div>The identification of critical nodes is fundamental to understanding complex network dynamics. Current Laplacian-based centrality measures predominantly build upon either the standard Laplacian matrix or distance Laplacian matrices. While effective, these approaches are limited to direct connections or geodesic paths, missing important higher-order structural information. This paper introduces Similarity Laplacian Centrality (SLC), which fundamentally redefines the Laplacian foundation by constructing a similarity Laplacian matrix based on Local Path index. Our key innovation lies in replacing the conventional adjacency/distance matrices with a similarity matrix that incorporates both direct and indirect relationships through paths of length 2 and 3. This allows SLC to capture functional similarities between nodes that extend beyond immediate neighborhood or shortest-path considerations. Within this novel framework, node importance is measured by the perturbation to the spectral energy of the Similarity Laplacian Matrix upon node removal. We provide theoretical guarantees and derive a computationally efficient closed-form solution that scales to large networks. Large-scale experiments conducted across multiple real-world networks demonstrate that the SLC method achieves strong performance in influence propagation, ranking accuracy, and discrimination capability across most datasets, effectively identifying critical nodes within complex networks.</div></div>","PeriodicalId":20152,"journal":{"name":"Physica A: Statistical Mechanics and its Applications","volume":"686 ","pages":"Article 131349"},"PeriodicalIF":3.1,"publicationDate":"2026-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147385749","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Influence maximization using combined community-level influence score 利用综合社区级影响力评分实现影响力最大化
IF 3.1 3区 物理与天体物理
Physica A: Statistical Mechanics and its Applications Pub Date : 2026-03-15 Epub Date: 2026-02-05 DOI: 10.1016/j.physa.2026.131367
Anil Tudu , Ardhendu Mandal , Debaditya Barman
{"title":"Influence maximization using combined community-level influence score","authors":"Anil Tudu ,&nbsp;Ardhendu Mandal ,&nbsp;Debaditya Barman","doi":"10.1016/j.physa.2026.131367","DOIUrl":"10.1016/j.physa.2026.131367","url":null,"abstract":"<div><div>Influence maximization (IM) is a NP-hard problem in the field of complex network with diverse applications, including viral marketing, information diffusion, and epidemic modeling. The goal is to identify a small set of influential nodes capable of initiating large-scale diffusion cascades. Although numerous IM algorithms have been developed to improve computational efficiency and output accuracy, they often face a trade-off between scalability and precision. Community-based IM methods address this challenge by exploiting the modular structure of networks to reduce the search space and enable parallel computation. This study proposes a novel community-based approach, Influence Maximization using Combined Community-level Influence Score (IMCCIS), which integrates both intra-community (local-level) and inter-community (global-level) influence scores. These scores are combined through a heuristic weighting scheme to identify nodes that effectively propagate influence within and across communities. Additionally, a penalty mechanism is introduced to mitigate overlapping influence among selected nodes. The proposed algorithm is inherently parallelizable, enhancing its suitability for large-scale networks. Comprehensive experiments on multiple real-world datasets demonstrate that IMCCIS achieves superior propagation performance and lower execution time compared to contemporary heuristic and community-based IM methods.</div></div>","PeriodicalId":20152,"journal":{"name":"Physica A: Statistical Mechanics and its Applications","volume":"686 ","pages":"Article 131367"},"PeriodicalIF":3.1,"publicationDate":"2026-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147385811","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
ϕ-Extropy complexity measure: Extensions and applications 外向复杂性度量:扩展和应用
IF 3.1 3区 物理与天体物理
Physica A: Statistical Mechanics and its Applications Pub Date : 2026-03-15 Epub Date: 2026-01-30 DOI: 10.1016/j.physa.2026.131278
Omid Kharazmi , Javier E. Contreras-Reyes , Cihat Erol , Femin Yalcin
{"title":"ϕ-Extropy complexity measure: Extensions and applications","authors":"Omid Kharazmi ,&nbsp;Javier E. Contreras-Reyes ,&nbsp;Cihat Erol ,&nbsp;Femin Yalcin","doi":"10.1016/j.physa.2026.131278","DOIUrl":"10.1016/j.physa.2026.131278","url":null,"abstract":"<div><div>In this paper, we first introduce the <span><math><mi>ϕ</mi></math></span>-extropy measure and the <span><math><mi>ϕ</mi></math></span>-extropy divergence (or relative <span><math><mi>ϕ</mi></math></span>-extropy) measure. We then explore their theoretical properties, highlighting that the <span><math><mi>ϕ</mi></math></span>-extropy encompasses the Gini–Simpson index as a special case. Furthermore, we establish the Bregman divergence in terms of two discrete complementary distributions and the Jensen-<span><math><mi>ϕ</mi></math></span>-extopy divergence. It is shown that the Jensen-<span><math><mi>ϕ</mi></math></span>-extropy divergence can be expressed as a mixture of the proposed Bregman divergences defined for two discrete complementary distributions. We also introduce two cumulative versions of the <span><math><mi>ϕ</mi></math></span>-extropy divergence: one defined in terms of the cumulative distribution function and the other defined in terms of the survival function. A mixture of these two cumulative versions of the <span><math><mi>ϕ</mi></math></span>-extropy divergence is proposed to demonstrate performance in real-world applications. Finally, we apply the proposed <span><math><mi>ϕ</mi></math></span>-extropy and mixture <span><math><mi>ϕ</mi></math></span>-extropy divergence to four cases. The first three cases are related to embedded time series representation used for the computation of permutation entropy, where proposed measures are considered for logistic and Chebyshev chaotic maps, and the impact of number of deaths produced by Covid-19 in India, the USA, and China during the period 2020–2022. The fourth case is related to a binary classification task using heart disease data and compares its performance against ten machine learning models, evaluating accuracy, precision, recall, and F1-score based on a 75%–25% train–test split of the heart disease dataset.</div></div>","PeriodicalId":20152,"journal":{"name":"Physica A: Statistical Mechanics and its Applications","volume":"686 ","pages":"Article 131278"},"PeriodicalIF":3.1,"publicationDate":"2026-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147385823","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Self-attention vector output similarities reveal how machines pay attention 自关注向量输出相似性揭示了机器如何关注
IF 3.1 3区 物理与天体物理
Physica A: Statistical Mechanics and its Applications Pub Date : 2026-03-15 Epub Date: 2026-02-04 DOI: 10.1016/j.physa.2026.131363
Tal Halevi , Yarden Tzach , Ronit D. Gross , Shalom Rosner , Ido Kanter
{"title":"Self-attention vector output similarities reveal how machines pay attention","authors":"Tal Halevi ,&nbsp;Yarden Tzach ,&nbsp;Ronit D. Gross ,&nbsp;Shalom Rosner ,&nbsp;Ido Kanter","doi":"10.1016/j.physa.2026.131363","DOIUrl":"10.1016/j.physa.2026.131363","url":null,"abstract":"<div><div>The self-attention mechanism has significantly advanced the field of natural language processing, facilitating the development of advanced language-learning machines. Although its utility is widely acknowledged, the precise mechanisms of self-attention underlying its advanced learning and the quantitative characterization of this learning process remains an open research question. This study introduces a new approach for quantifying information processing within the self-attention mechanism. The analysis conducted on the BERT-12 architecture reveals that, in the final layers, the attention map focuses on sentence separator tokens, suggesting a practical approach to text segmentation based on semantic features. Based on the vector space emerging from the self-attention heads, a context similarity matrix, measuring the scalar product between two token vectors was derived, revealing distinct context similarities between different token vector pairs within each head and layer. The findings demonstrated that different attention heads within an attention block focused on different linguistic characteristics, such as identifying token repetitions in a given text or recognizing a token of common appearance in the text and its surrounding context. This specialization is also reflected in the distribution of distances between token vectors with high context similarities as the architecture progresses. The initial attention layers exhibit substantially long-range context similarities; however, as the layers progress, a more short-range context similarity develops, culminating in a preference for attention heads to create strong context similarities within the same sentence. Finally, the behavior of individual heads was analyzed by examining the uniqueness of their most common tokens in their high context similarity elements. Each head tends to focus on a unique token from the text and builds similarity pairs centered around it. This methodology for quantifying the behavior of the vector space emerging from the self-attention layers offers a novel way to understand the dynamics of the self-attention mechanism with respect to its outputs.</div></div>","PeriodicalId":20152,"journal":{"name":"Physica A: Statistical Mechanics and its Applications","volume":"686 ","pages":"Article 131363"},"PeriodicalIF":3.1,"publicationDate":"2026-03-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147385727","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书