Machine Learning最新文献_第2页

Evaluating soccer match prediction models: a deep learning approach and feature optimization for gradient-boosted trees 评估足球比赛预测模型：梯度提升树的深度学习方法和特征优化

IF 7.5 3区计算机科学

Machine Learning Pub Date : 2024-08-23 DOI: 10.1007/s10994-024-06608-w

Calvin Yeung, Rory Bunker, Rikuhei Umemoto, Keisuke Fujii

{"title":"Evaluating soccer match prediction models: a deep learning approach and feature optimization for gradient-boosted trees","authors":"Calvin Yeung, Rory Bunker, Rikuhei Umemoto, Keisuke Fujii","doi":"10.1007/s10994-024-06608-w","DOIUrl":"https://doi.org/10.1007/s10994-024-06608-w","url":null,"abstract":"Machine learning models have become increasingly popular for predicting the results of soccer matches, however, the lack of publicly-available benchmark datasets has made model evaluation challenging. The 2023 Soccer Prediction Challenge required the prediction of match results first in terms of the exact goals scored by each team, and second, in terms of the probabilities for a win, draw, and loss. The original training set of matches and features, which was provided for the competition, was augmented with additional matches that were played between 4 April and 13 April 2023, representing the period after which the training set ended, but prior to the first matches that were to be predicted (upon which the performance was evaluated). A CatBoost model was employed using pi-ratings as the features, which were initially identified as the optimal choice for calculating the win/draw/loss probabilities. Notably, deep learning models have frequently been disregarded in this particular task. Therefore, in this study, we aimed to assess the performance of a deep learning model and determine the optimal feature set for a gradient-boosted tree model. The model was trained using the most recent 5 years of data, and three training and validation sets were used in a hyperparameter grid search. The results from the validation sets show that our model had strong performance and stability compared to previously published models from the 2017 Soccer Prediction Challenge for win/draw/loss prediction. Our model ranked 16th in the 2023 Soccer Prediction Challenge with RPS 0.2195.","PeriodicalId":49900,"journal":{"name":"Machine Learning","volume":"55 1","pages":""},"PeriodicalIF":7.5,"publicationDate":"2024-08-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142209733","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A cross-domain user association scheme based on graph attention networks with trajectory embedding 基于轨迹嵌入的图注意网络的跨域用户关联方案

IF 7.5 3区计算机科学

Machine Learning Pub Date : 2024-08-21 DOI: 10.1007/s10994-024-06613-z

Keqing Cen, Zhenghao Yang, Ze Wang, Minhong Dong

{"title":"A cross-domain user association scheme based on graph attention networks with trajectory embedding","authors":"Keqing Cen, Zhenghao Yang, Ze Wang, Minhong Dong","doi":"10.1007/s10994-024-06613-z","DOIUrl":"https://doi.org/10.1007/s10994-024-06613-z","url":null,"abstract":"With the widespread adoption of mobile internet, users generate vast amounts of location-based data across multiple social networking platforms. This data is valuable for applications such as personalized recommendations and targeted advertising. Accurately identifying users across different platforms enhances understanding of user behavior and preferences. To address the complexity of cross-domain user identification caused by varying check-in frequencies and data precision differences, we propose HTEGAT, a hierarchical trajectory embedding-based graph attention network model. HTEGAT addresses these issues by combining an Encoder and a Trajectory Identification module. The Encoder module, by integrating self-attention mechanisms with LSTM, can effectively extract location point-level features and accurately capture trajectory transition features, thereby accurately characterizing hierarchical temporal trajectories. Trajectory Identification module introduces trajectory distance-neighbor relationships and constructs an adjacency matrix based on these relationships. By utilizing attention weight coefficients in a graph attention network to capture similarities between trajectories, this approach reduces identification complexity while addressing the issue of dataset sparsity. Experiments on two cross-domain Location-Based Social Network (LBSN) datasets demonstrate that HTEGAT achieves higher hit rates with lower time complexity. On the Foursquare-Twitter dataset, HTEGAT significantly improved hit rates, surpassing state-of-the-art methods. On the Instagram-Twitter dataset, HTEGAT consistently outperformed contemporary models, showcasing its effectiveness and superiority.","PeriodicalId":49900,"journal":{"name":"Machine Learning","volume":"1 1","pages":""},"PeriodicalIF":7.5,"publicationDate":"2024-08-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142226444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A class sensitivity feature guided T-type generative model for noisy label classification 用于噪声标签分类的类敏感度特征引导 T 型生成模型

IF 7.5 3区计算机科学

Machine Learning Pub Date : 2024-08-20 DOI: 10.1007/s10994-024-06598-9

Yidi Bai, Hengjian Cui

{"title":"A class sensitivity feature guided T-type generative model for noisy label classification","authors":"Yidi Bai, Hengjian Cui","doi":"10.1007/s10994-024-06598-9","DOIUrl":"https://doi.org/10.1007/s10994-024-06598-9","url":null,"abstract":"Large-scale datasets inevitably contain noisy labels, which induces weak performance of deep neural networks (DNNs). Many existing methods focus on loss and regularization tricks, as well as characterizing and modelling differences between noisy and clean samples. However, taking advantage of information from different extents of distortion in latent feature space, is less explored and remains challenging. To solve this problem, we analyze characteristic distortion extents of different high-dimensional features, achieving the conclusion that features vary in their degree of deformation in their correlations with respect to categorical variables. Aforementioned disturbances on features not only reduce sensitivity and contribution of latent features to classification, but also bring obstacles into generating decision boundaries. To mitigate these issues, we propose class sensitivity feature extractor (CSFE) and T-type generative classifier (TGC). Based on the weighted Mahalanobis distance between conditional and unconditional cumulative distribution function after variance-stabilizing transformation, CSFE realizes high quality feature extraction through evaluating class-wise discrimination ability and sensitivity to classification. TGC introduces student-t estimator to clustering analysis in latent space, which is more robust in generating decision boundaries while maintaining equivalent efficiency. To alleviate the cost of retraining a whole DNN, we propose an ensemble model to simultaneously generate robust decision boundaries and train the DNN with the improved CSFE named SoftCSFE. Extensive experiments on three datasets, which are the RML2016.10a dataset, UCR Time Series Classification Archive dataset and a real-world dataset Clothing1M, show advantages of our methods.","PeriodicalId":49900,"journal":{"name":"Machine Learning","volume":"58 1","pages":""},"PeriodicalIF":7.5,"publicationDate":"2024-08-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142209736","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Weighting non-IID batches for out-of-distribution detection 对非 IID 批次进行加权，以检测配送外情况

IF 7.5 3区计算机科学

Machine Learning Pub Date : 2024-08-19 DOI: 10.1007/s10994-024-06605-z

Zhilin Zhao, Longbing Cao

{"title":"Weighting non-IID batches for out-of-distribution detection","authors":"Zhilin Zhao, Longbing Cao","doi":"10.1007/s10994-024-06605-z","DOIUrl":"https://doi.org/10.1007/s10994-024-06605-z","url":null,"abstract":"A standard network pretrained on in-distribution (ID) samples could make high-confidence predictions on out-of-distribution (OOD) samples, leaving the possibility of failing to distinguish ID and OOD samples in the test phase. To address this over-confidence issue, the existing methods improve the OOD sensitivity from modeling perspectives, i.e., retraining it by modifying training processes or objective functions. In contrast, this paper proposes a simple but effective method, namely Weighted Non-IID Batching (WNB), by adjusting batch weights. WNB builds on a key observation: increasing the batch size can improve the OOD detection performance. This is because a smaller batch size may make its batch samples more likely to be treated as non-IID from the assumed ID, i.e., associated with an OOD. This causes a network to provide high-confidence predictions for all samples from the OOD. Accordingly, WNB applies a weight function to weight each batch according to the discrepancy between batch samples and the entire training ID dataset. Specifically, the weight function is derived by minimizing the generalization error bound. It ensures that the weight function assigns larger weights to batches with smaller discrepancies and makes a trade-off between ID classification and OOD detection performance. Experimental results show that incorporating WNB into state-of-the-art OOD detection methods can further improve their performance.","PeriodicalId":49900,"journal":{"name":"Machine Learning","volume":"267 1","pages":""},"PeriodicalIF":7.5,"publicationDate":"2024-08-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142209734","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Understanding prediction discrepancies in classification 了解分类中的预测差异

IF 7.5 3区计算机科学

Machine Learning Pub Date : 2024-08-07 DOI: 10.1007/s10994-024-06557-4

Xavier Renard, Thibault Laugel, Marcin Detyniecki

引用次数: 0

Empirical Bayes linked matrix decomposition 经验贝叶斯关联矩阵分解

IF 7.5 3区计算机科学

Machine Learning Pub Date : 2024-08-07 DOI: 10.1007/s10994-024-06599-8

Eric F. Lock

{"title":"Empirical Bayes linked matrix decomposition","authors":"Eric F. Lock","doi":"10.1007/s10994-024-06599-8","DOIUrl":"https://doi.org/10.1007/s10994-024-06599-8","url":null,"abstract":"Data for several applications in diverse fields can be represented as multiple matrices that are linked across rows or columns. This is particularly common in molecular biomedical research, in which multiple molecular “omics” technologies may capture different feature sets (e.g., corresponding to rows in a matrix) and/or different sample populations (corresponding to columns). This has motivated a large body of work on integrative matrix factorization approaches that identify and decompose low-dimensional signal that is shared across multiple matrices or specific to a given matrix. We propose an empirical variational Bayesian approach to this problem that has several advantages over existing techniques, including the flexibility to accommodate shared signal over any number of row or column sets (i.e., bidimensional integration), an intuitive model-based objective function that yields appropriate shrinkage for the inferred signals, and a relatively efficient estimation algorithm with no tuning parameters. A general result establishes conditions for the uniqueness of the underlying decomposition for a broad family of methods that includes the proposed approach. For scenarios with missing data, we describe an associated iterative imputation approach that is novel for the single-matrix context and a powerful approach for “blockwise” imputation (in which an entire row or column is missing) in various linked matrix contexts. Extensive simulations show that the method performs very well under different scenarios with respect to recovering underlying low-rank signal, accurately decomposing shared and specific signals, and accurately imputing missing data. The approach is applied to gene expression and miRNA data from breast cancer tissue and normal breast tissue, for which it gives an informative decomposition of variation and outperforms alternative strategies for missing data imputation.","PeriodicalId":49900,"journal":{"name":"Machine Learning","volume":"24 1","pages":""},"PeriodicalIF":7.5,"publicationDate":"2024-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141941627","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Learning an adaptive forwarding strategy for mobile wireless networks: resource usage vs. latency 学习移动无线网络的自适应转发策略：资源使用与延迟

IF 7.5 3区计算机科学

Machine Learning Pub Date : 2024-08-07 DOI: 10.1007/s10994-024-06601-3

Victoria Manfredi, Alicia P. Wolfe, Xiaolan Zhang, Bing Wang

{"title":"Learning an adaptive forwarding strategy for mobile wireless networks: resource usage vs. latency","authors":"Victoria Manfredi, Alicia P. Wolfe, Xiaolan Zhang, Bing Wang","doi":"10.1007/s10994-024-06601-3","DOIUrl":"https://doi.org/10.1007/s10994-024-06601-3","url":null,"abstract":"Mobile wireless networks present several challenges for any learning system, due to uncertain and variable device movement, a decentralized network architecture, and constraints on network resources. In this work, we use deep reinforcement learning (DRL) to learn a scalable and generalizable forwarding strategy for such networks. We make the following contributions: (i) we use hierarchical RL to design DRL packet agents rather than device agents to capture the packet forwarding decisions that are made over time and improve training efficiency; (ii) we use relational features to ensure generalizability of the learned forwarding strategy to a wide range of network dynamics and enable offline training; and (iii) we incorporate both forwarding goals and network resource considerations into packet decision-making by designing a weighted reward function. Our results show that the forwarding strategy used by our DRL packet agent often achieves a similar delay per packet delivered as the oracle forwarding strategy and almost always outperforms all other strategies (including state-of-the-art strategies) in terms of delay, even on scenarios on which the DRL agent was not trained.","PeriodicalId":49900,"journal":{"name":"Machine Learning","volume":"79 1","pages":""},"PeriodicalIF":7.5,"publicationDate":"2024-08-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141941626","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A generic approach for reproducible model distillation 可重复模型提炼的通用方法

IF 7.5 3区计算机科学

Machine Learning Pub Date : 2024-08-05 DOI: 10.1007/s10994-024-06597-w

Yunzhe Zhou, Peiru Xu, Giles Hooker

{"title":"A generic approach for reproducible model distillation","authors":"Yunzhe Zhou, Peiru Xu, Giles Hooker","doi":"10.1007/s10994-024-06597-w","DOIUrl":"https://doi.org/10.1007/s10994-024-06597-w","url":null,"abstract":"Model distillation has been a popular method for producing interpretable machine learning. It uses an interpretable “student” model to mimic the predictions made by the black box “teacher” model. However, when the student model is sensitive to the variability of the data sets used for training even when keeping the teacher fixed, the corresponded interpretation is not reliable. Existing strategies stabilize model distillation by checking whether a large enough sample of pseudo-data is generated to reliably reproduce student models, but methods to do so have so far been developed separately for each specific class of student model. In this paper, we develop a generic approach for stable model distillation based on central limit theorem for the estimated fidelity of the student to the teacher. We start with a collection of candidate student models and search for candidates that reasonably agree with the teacher. Then we construct a multiple testing framework to select a sample size such that the consistent student model would be selected under different pseudo samples. We demonstrate the application of our proposed approach on three commonly used intelligible models: decision trees, falling rule lists and symbolic regression. Finally, we conduct simulation experiments on Mammographic Mass and Breast Cancer datasets and illustrate the testing procedure throughout a theoretical analysis with Markov process. The code is publicly available at https://github.com/yunzhe-zhou/GenericDistillation.","PeriodicalId":49900,"journal":{"name":"Machine Learning","volume":"23 1","pages":""},"PeriodicalIF":7.5,"publicationDate":"2024-08-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141941628","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Autoreplicative random forests with applications to missing value imputation 自复制随机森林在缺失值估算中的应用

IF 7.5 3区计算机科学

Machine Learning Pub Date : 2024-08-01 DOI: 10.1007/s10994-024-06584-1

Ekaterina Antonenko, Ander Carreño, Jesse Read

{"title":"Autoreplicative random forests with applications to missing value imputation","authors":"Ekaterina Antonenko, Ander Carreño, Jesse Read","doi":"10.1007/s10994-024-06584-1","DOIUrl":"https://doi.org/10.1007/s10994-024-06584-1","url":null,"abstract":"Missing values are a common problem in data science and machine learning. Removing instances with missing values is a straightforward workaround, but this can significantly hinder subsequent data analysis, particularly when features outnumber instances. There are a variety of methodologies proposed in the literature for imputing missing values. Denoising Autoencoders, for example, have been leveraged efficiently for imputation. However, neural network approaches have been relatively less effective on smaller datasets. In this work, we propose Autoreplicative Random Forests (ARF) as a multi-output learning approach, which we introduce in the context of a framework that may impute via either an iterative or procedural process. Experiments on several low- and high-dimensional datasets show that ARF is computationally efficient and exhibits better imputation performance than its competitors, including neural network approaches. In order to provide statistical analysis and mathematical background to the proposed missing value imputation framework, we also propose probabilistic ARFs, where the confidence values are provided over different imputation hypotheses, therefore maximizing the utility of such a framework in a machine-learning pipeline targeting predictive performance.","PeriodicalId":49900,"journal":{"name":"Machine Learning","volume":"219 1","pages":""},"PeriodicalIF":7.5,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141886519","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Describing group evolution in temporal data using multi-faceted events 利用多方面事件描述时间数据中的群体演变

IF 7.5 3区计算机科学

Machine Learning Pub Date : 2024-08-01 DOI: 10.1007/s10994-024-06600-4

Andrea Failla, Rémy Cazabet, Giulio Rossetti, Salvatore Citraro

{"title":"Describing group evolution in temporal data using multi-faceted events","authors":"Andrea Failla, Rémy Cazabet, Giulio Rossetti, Salvatore Citraro","doi":"10.1007/s10994-024-06600-4","DOIUrl":"https://doi.org/10.1007/s10994-024-06600-4","url":null,"abstract":"Groups—such as clusters of points or communities of nodes—are fundamental when addressing various data mining tasks. In temporal data, the predominant approach for characterizing group evolution has been through the identification of “events”. However, the events usually described in the literature, e.g., shrinks/growths, splits/merges, are often arbitrarily defined, creating a gap between such theoretical/predefined types and real-data group observations. Moving beyond existing taxonomies, we think of events as “archetypes” characterized by a unique combination of quantitative dimensions that we call “facets”. Group dynamics are defined by their position within the facet space, where archetypal events occupy extremities. Thus, rather than enforcing strict event types, our approach can allow for hybrid descriptions of dynamics involving group proximity to multiple archetypes. We apply our framework to evolving groups from several face-to-face interaction datasets, showing it enables richer, more reliable characterization of group dynamics with respect to state-of-the-art methods, especially when the groups are subject to complex relationships. Our approach also offers intuitive solutions to common tasks related to dynamic group analysis, such as choosing an appropriate aggregation scale, quantifying partition stability, and evaluating event quality.","PeriodicalId":49900,"journal":{"name":"Machine Learning","volume":"78 1","pages":""},"PeriodicalIF":7.5,"publicationDate":"2024-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141872948","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0