Machine Learning Science and Technology最新文献_第5页

Concept graph embedding models for enhanced accuracy and interpretability 概念图嵌入模型，提高准确性和可解释性

IF 6.8 2区物理与天体物理

Machine Learning Science and Technology Pub Date : 2024-08-13 DOI: 10.1088/2632-2153/ad6ad2

Sangwon Kim, Byoung Chul Ko

{"title":"Concept graph embedding models for enhanced accuracy and interpretability","authors":"Sangwon Kim, Byoung Chul Ko","doi":"10.1088/2632-2153/ad6ad2","DOIUrl":"https://doi.org/10.1088/2632-2153/ad6ad2","url":null,"abstract":"In fields requiring high accountability, it is necessary to understand how deep-learning models make decisions when analyzing the causes of image classification. Concept-based interpretation methods have recently been introduced to reveal the internal mechanisms of deep learning models using high-level concepts. However, such methods are constrained by a trade-off between accuracy and interpretability. For instance, in real-world environments, unlike in well-curated training data, the accurate prediction of expected concepts becomes a challenge owing to the various distortions and complexities introduced by different objects. To overcome this tradeoff, we propose concept graph embedding models (CGEM), reflecting the complex dependencies and structures among concepts through the learning of mutual directionalities. The concept graph convolutional neural network (Concept GCN), a downstream task of CGEM, differs from previous methods that solely determine the presence of concepts because it performs a final classification based on the relationships between con- cepts learned through graph embedding. This process endows the model with high resilience even in the presence of incorrect concepts. In addition, we utilize a deformable bipartite GCN for object- centric concept encoding in the earlier stages, which enhances the homogeneity of the concepts. The experimental results show that, based on deformable concept encoding, the CGEM mitigates the trade-off between task accuracy and interpretability. Moreover, it was confirmed that this approach allows the model to increase the resilience and interpretability while maintaining robustness against various real-world concept distortions and incorrect concept interventions. Our code is available at <ext-link ext-link-type=\"uri\" xlink:href=\"https://github.com/jumpsnack/cgem\">https://github.com/jumpsnack/cgem</ext-link>.","PeriodicalId":33757,"journal":{"name":"Machine Learning Science and Technology","volume":"69 1","pages":""},"PeriodicalIF":6.8,"publicationDate":"2024-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142197707","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Virtual reality for understanding artificial-intelligence-driven scientific discovery with an application in quantum optics 虚拟现实技术在量子光学中的应用：了解人工智能驱动的科学发现

IF 6.8 2区物理与天体物理

Machine Learning Science and Technology Pub Date : 2024-08-13 DOI: 10.1088/2632-2153/ad5fdb

Philipp Schmidt, Sören Arlt, Carlos Ruiz-Gonzalez, Xuemei Gu, Carla Rodríguez, Mario Krenn

{"title":"Virtual reality for understanding artificial-intelligence-driven scientific discovery with an application in quantum optics","authors":"Philipp Schmidt, Sören Arlt, Carlos Ruiz-Gonzalez, Xuemei Gu, Carla Rodríguez, Mario Krenn","doi":"10.1088/2632-2153/ad5fdb","DOIUrl":"https://doi.org/10.1088/2632-2153/ad5fdb","url":null,"abstract":"Generative Artificial Intelligence (AI) models can propose solutions to scientific problems beyond human capability. To truly make conceptual contributions, researchers need to be capable of understanding the AI-generated structures and extracting the underlying concepts and ideas. When algorithms provide little explanatory reasoning alongside the output, scientists have to reverse-engineer the fundamental insights behind proposals based solely on examples. This task can be challenging as the output is often highly complex and thus not immediately accessible to humans. In this work we show how transferring part of the analysis process into an immersive virtual reality (VR) environment can assist researchers in developing an understanding of AI-generated solutions. We demonstrate the usefulness of VR in finding interpretable configurations of abstract graphs, representing Quantum Optics experiments. Thereby, we can manually discover new generalizations of AI-discoveries as well as new understanding in experimental quantum optics. Furthermore, it allows us to customize the search space in an informed way—as a human-in-the-loop—to achieve significantly faster subsequent discovery iterations. As concrete examples, with this technology, we discover a new resource-efficient 3-dimensional entanglement swapping scheme, as well as a 3-dimensional 4-particle Greenberger–Horne–Zeilinger-state analyzer. Our results show the potential of VR to enhance a researcher’s ability to derive knowledge from graph-based generative AI. This type of AI is a widely used abstract data representation in various scientific fields.","PeriodicalId":33757,"journal":{"name":"Machine Learning Science and Technology","volume":"15 1","pages":""},"PeriodicalIF":6.8,"publicationDate":"2024-08-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142197704","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Transfer learning with generative models for object detection on limited datasets 利用生成模型在有限数据集上进行物体检测的迁移学习

IF 6.8 2区物理与天体物理

Machine Learning Science and Technology Pub Date : 2024-08-11 DOI: 10.1088/2632-2153/ad65b5

M Paiano, S Martina, C Giannelli and F Caruso

{"title":"Transfer learning with generative models for object detection on limited datasets","authors":"M Paiano, S Martina, C Giannelli and F Caruso","doi":"10.1088/2632-2153/ad65b5","DOIUrl":"https://doi.org/10.1088/2632-2153/ad65b5","url":null,"abstract":"The availability of data is limited in some fields, especially for object detection tasks, where it is necessary to have correctly labeled bounding boxes around each object. A notable example of such data scarcity is found in the domain of marine biology, where it is useful to develop methods to automatically detect submarine species for environmental monitoring. To address this data limitation, the state-of-the-art machine learning strategies employ two main approaches. The first involves pretraining models on existing datasets before generalizing to the specific domain of interest. The second strategy is to create synthetic datasets specifically tailored to the target domain using methods like copy-paste techniques or ad-hoc simulators. The first strategy often faces a significant domain shift, while the second demands custom solutions crafted for the specific task. In response to these challenges, here we propose a transfer learning framework that is valid for a generic scenario. In this framework, generated images help to improve the performances of an object detector in a few-real data regime. This is achieved through a diffusion-based generative model that was pretrained on large generic datasets. With respect to the state-of-the-art, we find that it is not necessary to fine tune the generative model on the specific domain of interest. We believe that this is an important advance because it mitigates the labor-intensive task of manual labeling the images in object detection tasks. We validate our approach focusing on fishes in an underwater environment, and on the more common domain of cars in an urban setting. Our method achieves detection performance comparable to models trained on thousands of images, using only a few hundreds of input data. Our results pave the way for new generative AI-based protocols for machine learning applications in various domains, for instance ranging from geophysics to biology and medicine.","PeriodicalId":33757,"journal":{"name":"Machine Learning Science and Technology","volume":"192 1","pages":""},"PeriodicalIF":6.8,"publicationDate":"2024-08-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141931252","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Trainability issues in quantum policy gradients 量子政策梯度的可训练性问题

IF 6.8 2区物理与天体物理

Machine Learning Science and Technology Pub Date : 2024-08-05 DOI: 10.1088/2632-2153/ad6830

André Sequeira, Luis Paulo Santos and Luis Soares Barbosa

引用次数: 0

Molecular relaxation by reverse diffusion with time step prediction 反向扩散的分子弛豫与时间步长预测

IF 6.8 2区物理与天体物理

Machine Learning Science and Technology Pub Date : 2024-08-05 DOI: 10.1088/2632-2153/ad652c

Khaled Kahouli, Stefaan Simon Pierre Hessmann, Klaus-Robert Müller, Shinichi Nakajima, Stefan Gugler and Niklas Wolf Andreas Gebauer

{"title":"Molecular relaxation by reverse diffusion with time step prediction","authors":"Khaled Kahouli, Stefaan Simon Pierre Hessmann, Klaus-Robert Müller, Shinichi Nakajima, Stefan Gugler and Niklas Wolf Andreas Gebauer","doi":"10.1088/2632-2153/ad652c","DOIUrl":"https://doi.org/10.1088/2632-2153/ad652c","url":null,"abstract":"Molecular relaxation, finding the equilibrium state of a non-equilibrium structure, is an essential component of computational chemistry to understand reactivity. Classical force field (FF) methods often rely on insufficient local energy minimization, while neural network FF models require large labeled datasets encompassing both equilibrium and non-equilibrium structures. As a remedy, we propose MoreRed, molecular relaxation by reverse diffusion, a conceptually novel and purely statistical approach where non-equilibrium structures are treated as noisy instances of their corresponding equilibrium states. To enable the denoising of arbitrarily noisy inputs via a generative diffusion model, we further introduce a novel diffusion time step predictor. Notably, MoreRed learns a simpler pseudo potential energy surface (PES) instead of the complex physical PES. It is trained on a significantly smaller, and thus computationally cheaper, dataset consisting of solely unlabeled equilibrium structures, avoiding the computation of non-equilibrium structures altogether. We compare MoreRed to classical FFs, equivariant neural network FFs trained on a large dataset of equilibrium and non-equilibrium data, as well as a semi-empirical tight-binding model. To assess this quantitatively, we evaluate the root-mean-square deviation between the found equilibrium structures and the reference equilibrium structures as well as their energies.","PeriodicalId":33757,"journal":{"name":"Machine Learning Science and Technology","volume":"303 1","pages":""},"PeriodicalIF":6.8,"publicationDate":"2024-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141931253","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multi-perspective feedback-attention coupling model for continuous-time dynamic graphs 连续时间动态图的多视角反馈-关注耦合模型

IF 6.8 2区物理与天体物理

Machine Learning Science and Technology Pub Date : 2024-08-04 DOI: 10.1088/2632-2153/ad66af

Xiaobo Zhu, Yan Wu, Jin Che, Chao Wang, Liying Wang and Zhanheng Chen

{"title":"Multi-perspective feedback-attention coupling model for continuous-time dynamic graphs","authors":"Xiaobo Zhu, Yan Wu, Jin Che, Chao Wang, Liying Wang and Zhanheng Chen","doi":"10.1088/2632-2153/ad66af","DOIUrl":"https://doi.org/10.1088/2632-2153/ad66af","url":null,"abstract":"Representation learning over graph networks has recently gained popularity, with many models showing promising results. However, several challenges remain: (1) most methods are designed for static or discrete-time dynamic graphs; (2) existing continuous-time dynamic graph algorithms focus on a single evolving perspective; and (3) many continuous-time dynamic graph approaches necessitate numerous temporal neighbors to capture long-term dependencies. In response, this paper introduces a Multi-Perspective Feedback-Attention Coupling (MPFA) model. MPFA incorporates information from both evolving and original perspectives to effectively learn the complex dynamics of dynamic graph evolution processes. The evolving perspective considers the current state of historical interaction events of nodes and uses a temporal attention module to aggregate current state information. This perspective also makes it possible to capture long-term dependencies of nodes using a small number of temporal neighbors. Meanwhile, the original perspective utilizes a feedback attention module with growth characteristic coefficients to aggregate the original state information of node interactions. Experimental results on one dataset organized by ourselves and seven public datasets validate the effectiveness and competitiveness of our proposed model.","PeriodicalId":33757,"journal":{"name":"Machine Learning Science and Technology","volume":"14 1","pages":""},"PeriodicalIF":6.8,"publicationDate":"2024-08-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141968649","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Towards robust data-driven automated recovery of symbolic conservation laws from limited data 从有限数据中实现稳健的数据驱动自动恢复符号守恒定律

IF 6.8 2区物理与天体物理

Machine Learning Science and Technology Pub Date : 2024-08-04 DOI: 10.1088/2632-2153/ad6390

Tracey Oellerich and Maria Emelianenko

引用次数: 0

Coincident learning for unsupervised anomaly detection of scientific instruments 用于科学仪器无监督异常检测的巧合学习

IF 6.8 2区物理与天体物理

Machine Learning Science and Technology Pub Date : 2024-08-04 DOI: 10.1088/2632-2153/ad64a6

Ryan Humble, Zhe Zhang, Finn O’Shea, Eric Darve and Daniel Ratner

{"title":"Coincident learning for unsupervised anomaly detection of scientific instruments","authors":"Ryan Humble, Zhe Zhang, Finn O’Shea, Eric Darve and Daniel Ratner","doi":"10.1088/2632-2153/ad64a6","DOIUrl":"https://doi.org/10.1088/2632-2153/ad64a6","url":null,"abstract":"Anomaly detection is an important task for complex scientific experiments and other complex systems (e.g. industrial facilities, manufacturing), where failures in a sub-system can lead to lost data, poor performance, or even damage to components. While scientific facilities generate a wealth of data, labeled anomalies may be rare (or even nonexistent), and expensive to acquire. Unsupervised approaches are therefore common and typically search for anomalies either by distance or density of examples in the input feature space (or some associated low-dimensional representation). This paper presents a novel approach called coincident learning for anomaly detection (CoAD), which is specifically designed for multi-modal tasks and identifies anomalies based on coincident behavior across two different slices of the feature space. We define an unsupervised metric, , out of analogy to the supervised classification Fβ statistic. CoAD uses to train an anomaly detection algorithm on unlabeled data, based on the expectation that anomalous behavior in one feature slice is coincident with anomalous behavior in the other. The method is illustrated using a synthetic outlier data set and a MNIST-based image data set, and is compared to prior state-of-the-art on two real-world tasks: a metal milling data set and our motivating task of identifying RF station anomalies in a particle accelerator.","PeriodicalId":33757,"journal":{"name":"Machine Learning Science and Technology","volume":"76 1","pages":""},"PeriodicalIF":6.8,"publicationDate":"2024-08-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141931255","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

OmniJet-α: the first cross-task foundation model for particle physics OmniJet-α：首个用于粒子物理学的跨任务基础模型

IF 6.8 2区物理与天体物理

Machine Learning Science and Technology Pub Date : 2024-08-01 DOI: 10.1088/2632-2153/ad66ad

Joschka Birk, Anna Hallin and Gregor Kasieczka

{"title":"OmniJet-α: the first cross-task foundation model for particle physics","authors":"Joschka Birk, Anna Hallin and Gregor Kasieczka","doi":"10.1088/2632-2153/ad66ad","DOIUrl":"https://doi.org/10.1088/2632-2153/ad66ad","url":null,"abstract":"Foundation models are multi-dataset and multi-task machine learning methods that once pre-trained can be fine-tuned for a large variety of downstream applications. The successful development of such general-purpose models for physics data would be a major breakthrough as they could improve the achievable physics performance while at the same time drastically reduce the required amount of training time and data. We report significant progress on this challenge on several fronts. First, a comprehensive set of evaluation methods is introduced to judge the quality of an encoding from physics data into a representation suitable for the autoregressive generation of particle jets with transformer architectures (the common backbone of foundation models). These measures motivate the choice of a higher-fidelity tokenization compared to previous works. Finally, we demonstrate transfer learning between an unsupervised problem (jet generation) and a classic supervised task (jet tagging) with our new OmniJet-α model. This is the first successful transfer between two different and actively studied classes of tasks and constitutes a major step in the building of foundation models for particle physics.","PeriodicalId":33757,"journal":{"name":"Machine Learning Science and Technology","volume":"81 1","pages":""},"PeriodicalIF":6.8,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141885723","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Efficient Bayesian inference using physics-informed invertible neural networks for inverse problems 利用物理信息可逆神经网络对逆问题进行高效贝叶斯推理

IF 6.8 2区物理与天体物理

Machine Learning Science and Technology Pub Date : 2024-07-22 DOI: 10.1088/2632-2153/ad5f74

Xiaofei Guan, Xintong Wang, Hao Wu, Zihao Yang and Peng Yu

{"title":"Efficient Bayesian inference using physics-informed invertible neural networks for inverse problems","authors":"Xiaofei Guan, Xintong Wang, Hao Wu, Zihao Yang and Peng Yu","doi":"10.1088/2632-2153/ad5f74","DOIUrl":"https://doi.org/10.1088/2632-2153/ad5f74","url":null,"abstract":"This paper presents an innovative approach to tackle Bayesian inverse problems using physics-informed invertible neural networks (PI-INN). Serving as a neural operator model, PI-INN employs an invertible neural network (INN) to elucidate the relationship between the parameter field and the solution function in latent variable spaces. Specifically, the INN decomposes the latent variable of the parameter field into two distinct components: the expansion coefficients that represent the solution to the forward problem, and the noise that captures the inherent uncertainty associated with the inverse problem. Through precise estimation of the forward mapping and preservation of statistical independence between expansion coefficients and latent noise, PI-INN offers an accurate and efficient generative model for resolving Bayesian inverse problems, even in the absence of labeled data. For a given solution function, PI-INN can provide tractable and accurate estimates of the posterior distribution of the underlying parameter field. Moreover, capitalizing on the INN’s characteristics, we propose a novel independent loss function to effectively ensure the independence of the INN’s decomposition results. The efficacy and precision of the proposed PI-INN are demonstrated through a series of numerical experiments.","PeriodicalId":33757,"journal":{"name":"Machine Learning Science and Technology","volume":"214 1","pages":""},"PeriodicalIF":6.8,"publicationDate":"2024-07-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141753973","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"物理与天体物理","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0