ACM Transactions on Intelligent Systems and Technology (TIST)最新文献

{"title":"Federated Multi-task Graph Learning","authors":"Yijing Liu, Dongming Han, Jianwei Zhang, Haiyang Zhu, Mingliang Xu, Wei Chen","doi":"10.1145/3527622","DOIUrl":"https://doi.org/10.1145/3527622","url":null,"abstract":"Distributed processing and analysis of large-scale graph data remain challenging because of the high-level discrepancy among graphs. This study investigates a novel subproblem: the distributed multi-task learning on the graph, which jointly learns multiple analysis tasks from decentralized graphs. We propose a federated multi-task graph learning (FMTGL) framework to solve the problem within a privacy-preserving and scalable scheme. Its core is an innovative data-fusion mechanism and a low-latency distributed optimization method. The former captures multi-source data relatedness and generates universal task representation for local task analysis. The latter enables the quick update of our framework with gradients sparsification and tree-based aggregation. As a theoretical result, the proposed optimization method has a convergence rate interpolates between ( mathcal {O}(1/T) ) and ( mathcal {O}(1/sqrt {T}) ) , up to logarithmic terms. Unlike previous studies, our work analyzes the convergence behavior with adaptive stepsize selection and non-convex assumption. Experimental results on three graph datasets verify the effectiveness and scalability of FMTGL.","PeriodicalId":123526,"journal":{"name":"ACM Transactions on Intelligent Systems and Technology (TIST)","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121114999","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"FLeet: Online Federated Learning via Staleness Awareness and Performance Prediction","authors":"Georgios Damaskinos, R. Guerraoui, Anne-Marie Kermarrec, Vlad Nitu, Rhicheek Patra, Francois Taiani","doi":"10.1145/3527621","DOIUrl":"https://doi.org/10.1145/3527621","url":null,"abstract":"Federated learning (FL) is very appealing for its privacy benefits: essentially, a global model is trained with updates computed on mobile devices while keeping the data of users local. Standard FL infrastructures are however designed to have no energy or performance impact on mobile devices, and are therefore not suitable for applications that require frequent (online) model updates, such as news recommenders. This article presents FLeet, the first Online FL system, acting as a middleware between the Android operating system and the machine learning application. FLeet combines the privacy of Standard FL with the precision of online learning thanks to two core components: (1) I-Prof, a new lightweight profiler that predicts and controls the impact of learning tasks on mobile devices, and (2) AdaSGD, a new adaptive learning algorithm that is resilient to delayed updates. Our extensive evaluation shows that Online FL, as implemented by FLeet, can deliver a 2.3× quality boost compared to Standard FL while only consuming 0.036% of the battery per day. I-Prof can accurately control the impact of learning tasks by improving the prediction accuracy by up to 3.6× in terms of computation time, and by up to 19× in terms of energy. AdaSGD outperforms alternative FL approaches by 18.4% in terms of convergence speed on heterogeneous data.","PeriodicalId":123526,"journal":{"name":"ACM Transactions on Intelligent Systems and Technology (TIST)","volume":"7 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122173224","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Efficient and Effective Similar Subtrajectory Search: A Spatial-aware Comprehension Approach","authors":"Liwei Deng, Hao-Lun Sun, Rui Sun, Yan Zhao, Han Su","doi":"10.1145/3456723","DOIUrl":"https://doi.org/10.1145/3456723","url":null,"abstract":"Although many applications take subtrajectories as basic units for analysis, there is little research on the similar subtrajectory search problem aiming to return a portion of a trajectory (i.e., subtrajectory), which is the most similar to a query trajectory. We find that in some special cases, when a grid-based metric is used, this problem can be formulated as a reading comprehension problem, which has been studied extensively in the field of natural language processing (NLP). By this formulation, we can obtain faster models with better performance than existing methods. However, due to the difference between natural language and trajectory (e.g., spatial relationship), it is impossible to directly apply NLP models to this problem. Therefore, we propose a Similar Subtrajectory Search with a Graph Neural Networks framework. This framework contains four modules including a spatial-aware grid embedding module, a trajectory embedding module, a query-context trajectory fusion module, and a span prediction module. Specifically, in the spatial-aware grid embedding module, the spatial-based grid adjacency is constructed and delivered to the graph neural network to learn spatial-aware grid embedding. The trajectory embedding module aims to model the sequential information of trajectories. The purpose of the query-context trajectory fusion module is to fuse the information of the query trajectory to each grid of the context trajectories. Finally, the span prediction module aims to predict the start and the end of a subtrajectory for the context trajectory, which is the most similar to the query trajectory. We conduct comprehensive experiments on two real world datasets, where the proposed framework outperforms the state-of-the-art baselines consistently and significantly.","PeriodicalId":123526,"journal":{"name":"ACM Transactions on Intelligent Systems and Technology (TIST)","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"117046640","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"GPSClean: A Framework for Cleaning and Repairing GPS Data","authors":"Cheng-Hung Fang, Feng Wang, Bin Yao, Jianqiu Xu","doi":"10.1145/3469088","DOIUrl":"https://doi.org/10.1145/3469088","url":null,"abstract":"The rise of GPS-equipped mobile devices has led to the emergence of big trajectory data. The collected raw data usually contain errors and anomalies information caused by device failure, sensor error, and environment influence. Low-quality data fails to support application requirements and therefore raw data will be comprehensively cleaned before usage. Existing methods are suboptimal to detect GPS data errors and do the repairing. To solve the problem, we propose a framework called GPSClean to analyze the anomalies data and develop effective methods to repair the data. There are primarily four modules in GPSClean: (i) data preprocessing, (ii) data filling, (iii) data repairing, and (iv) data conversion. For (i), we propose an approach named MDSort (Maximum Disorder Sorting) to efficiently solve the issue of data disorder. For (ii), we propose a method named NNF (Nearest Neighbor Filling) to fill missing data. For (iii), we design an approach named RCSWS (Range Constraints and Sliding Window Statistics) to repair anomalies and also improve the accuracy of data repairing by mak7ing use of driving direction. We use 45 million real trajectory data to evaluate our proposal in a prototype database system SECONDO. Experimental results show that the accuracy of RCSWS is three times higher than an alternative method SCREEN and nearly an order of magnitude higher than an alternative method EWMA.","PeriodicalId":123526,"journal":{"name":"ACM Transactions on Intelligent Systems and Technology (TIST)","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-04-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130266481","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Self-supervised Short-text Modeling through Auxiliary Context Generation","authors":"Nurendra Choudhary, C. Aggarwal","doi":"10.1145/3511712","DOIUrl":"https://doi.org/10.1145/3511712","url":null,"abstract":"Short text is ambiguous and often relies predominantly on the domain and context at hand in order to attain semantic relevance. Existing classification models perform poorly on short text due to data sparsity and inadequate context. Auxiliary context, which can often provide sufficient background regarding the domain, is typically available in several application scenarios. While some of the existing works aim to leverage real-world knowledge to enhance short-text representations, they fail to place appropriate emphasis on the auxiliary context. Such models do not harness the full potential of the available context in auxiliary sources. To address this challenge, we reformulate short-text classification as a dual channel self-supervised learning problem (that leverages auxiliary context) with a generation network and a corresponding prediction model. We propose a self-supervised framework, Pseudo-Auxiliary Context generation network for Short-text Modeling (PACS), to comprehensively leverage auxiliary context and it is jointly learned with a prediction network in an end-to-end manner. Our PACS model consists of two sub-networks: a Context Generation Network (CGN) that models the auxiliary context’s distribution and a Prediction Network (PN) to map the short-text features and auxiliary context distribution to the final class label. Our experimental results on diverse datasets demonstrate that PACS outperforms formidable state-of-the-art baselines. We also demonstrate the performance of our model on cold-start scenarios (where contextual information is non-existent) during prediction. Furthermore, we perform interpretability and ablation studies to analyze various representational features captured by our model and the individual contribution of its modules to the overall performance of PACS, respectively.","PeriodicalId":123526,"journal":{"name":"ACM Transactions on Intelligent Systems and Technology (TIST)","volume":"16 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131432044","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Privacy Preservation for Trajectory Publication Based on Differential Privacy","authors":"Lin Yao, Zhenyu Chen, Haibo Hu, Guowei Wu, Bin Wu","doi":"10.1145/3474839","DOIUrl":"https://doi.org/10.1145/3474839","url":null,"abstract":"With the proliferation of location-aware devices, trajectory data have been used widely in real-life applications. However, trajectory data are often associated with sensitive labels, such as users’ purchase transactions and planned activities. As such, inappropriate sharing or publishing of these data could threaten users’ privacy, especially when an adversary has sufficient background knowledge about a trajectory through other data sources, such as social media (check-in tags). Though differential privacy has been used to address the privacy of trajectory data, no existing method can protect the privacy of both trajectory data and sensitive labels. In this article, we propose a comprehensive trajectory publishing algorithm with three effective procedures. First, we apply density-based clustering to determine hotspots and outliers and then blur their locations by generalization. Second, we propose a graph-based model to efficiently capture the relationship among sensitive labels and trajectory points in all records and leverage Laplace noise to achieve differential privacy. Finally, we generate and publish trajectories by traversing and updating this graph until we travel all vertexes. Our experiments on synthetic and real-life datasets demonstrate that our algorithm effectively protects the privacy of both sensitive labels and location data in trajectory publication. Compared with existing works on trajectory publishing, our algorithm can also achieve higher data utility.","PeriodicalId":123526,"journal":{"name":"ACM Transactions on Intelligent Systems and Technology (TIST)","volume":"133 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124137845","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Introduction to the Special Issue on Deep Learning for Spatio-Temporal Data: Part 2","authors":"Senzhang Wang, Junbo Zhang, Yanjie Fu, Yong Li","doi":"10.1145/3510023","DOIUrl":"https://doi.org/10.1145/3510023","url":null,"abstract":"framework that explicitly models the topological skeleton of a terrain surface with a contour tree from com-putational topology, which is guided by the physical constraint. Their framework consists of two neural networks: a convolutional neural network (CNN) to learn spatial contextual features on a 2D image grid and a graph neural network (GNN) to learn the statistical distribution of physics-guided spatial topological dependency on the contour tree. The two models are co-trained via variational EM. Evaluations on the real-world flood mapping datasets show that the proposed models outperform baseline methods in classification accuracy, especially when training labels are limited.Inthe article titled “Make More Connections: Urban Traffic Flow Forecasting with Spatiotemporal Adaptive Gated Graph Convolution Network,” Lu et al. consider constructing the road network as a dynamic weighted graph through the attention mechanism to describe and capture the dynamic spatio-temporal correlation, and they aim to seek both spatial neighbors and semantic neighbors to make more connections between road nodes. They propose a novel Spatio-temporal Adaptive Gated Graph Convolution Network (STAG-GCN) to predict traffic conditions for several time steps ahead. STAG-GCN mainly consists of two major components: (1) multivari-ate self-attention Temporal Convolution Network (TCN) is utilized to capture local and long-range temporal dependencies across recent, daily periodic and weekly periodic observations, and (2) mix-hop AG-GCN extracts selective spatial and semantic dependencies within multi-layer stack-ing through an adaptive graph gating mechanism and mix-hop propagation mechanism. The out-puts of different components are weighted fused to generate the final prediction results. Extensive experiments on two real-world large-scale urban traffic datasets have verified the effectiveness of their proposed approach.","PeriodicalId":123526,"journal":{"name":"ACM Transactions on Intelligent Systems and Technology (TIST)","volume":"93 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115730453","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"CrimeTensor: Fine-Scale Crime Prediction via Tensor Learning with Spatiotemporal Consistency","authors":"Weichao Liang, Zhiang Wu, Z. Li, Yong Ge","doi":"10.1145/3501807","DOIUrl":"https://doi.org/10.1145/3501807","url":null,"abstract":"Crime poses a major threat to human life and property, which has been recognized as one of the most crucial problems in our society. Predicting the number of crime incidents in each region of a city before they happen is of great importance to fight against crime. There has been a great deal of research focused on crime prediction, ranging from introducing diversified data sources to exploring various prediction models. However, most of the existing approaches fail to offer fine-scale prediction results and take little notice of the intricate spatial-temporal-categorical correlations contained in crime incidents. In this article, we propose a tailor-made framework called CrimeTensor to predict the number of crime incidents belonging to different categories within each target region via tensor learning with spatiotemporal consistency. In particular, we model the crime data as a tensor and present an objective function which tries to take full advantage of the spatial, temporal, and categorical correlations contained in crime incidents. Moreover, a well-designed optimization algorithm which transforms the objective into a compact form and then applies CP decomposition to find the optimal solution is elaborated to solve the objective function. Furthermore, we develop an enhanced framework which takes a set of pre-selected regions to conduct prediction so as to further improve the computational efficiency of the optimization algorithm. Finally, extensive experiments are performed on both proprietary and public datasets and our framework significantly outperforms all the baselines in terms of each evaluation metric.","PeriodicalId":123526,"journal":{"name":"ACM Transactions on Intelligent Systems and Technology (TIST)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-26","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126618876","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"INN: An Interpretable Neural Network for AI Incubation in Manufacturing","authors":"Xiaoyu Chen, Yingyan Zeng, Sungku Kang, R. Jin","doi":"10.1145/3519313","DOIUrl":"https://doi.org/10.1145/3519313","url":null,"abstract":"Both artificial intelligence (AI) and domain knowledge from human experts play an important role in manufacturing decision making. Smart manufacturing emphasizes a fully automated data-driven decision-making; however, the AI incubation process involves human experts to enhance AI systems by integrating domain knowledge for modeling, data collection and annotation, and feature extraction. Such an AI incubation process not only enhances the domain knowledge discovery but also improves the interpretability and trustworthiness of AI methods. In this article, we focus on the knowledge transfer from human experts to a supervised learning problem by learning domain knowledge as interpretable features and rules, which can be used to construct rule-based systems to support manufacturing decision making, such as process modeling and quality inspection. Although many advanced statistical and machine learning methods have shown promising modeling accuracy and efficiency, rule-based systems are still highly preferred and widely adopted due to their interpretability for human experts to comprehend. However, most of the existing rule-based systems are constructed based on deterministic human-crafted rules, whose parameters, such as thresholds of decision rules, are suboptimal. Yet the machine learning methods, such as tree models or neural networks, can learn a decision rule based structure without much interpretation or agreement with domain knowledge. Therefore, the traditional machine learning models and human experts’ domain knowledge cannot be directly improved by learning from data. In this research, we propose an interpretable neural network (INN) model with a center-adjustable sigmoid activation function to efficiently optimize the rule-based systems. Using the rule-based system from domain knowledge to regulate the INN architecture not only improves the prediction accuracy with optimized parameters but also ensures the interpretability by adopting the interpretable rule-based systems from domain knowledge. The proposed INN will be effective for supervised learning problems when rule-based systems are available. The merits of the INN model are demonstrated via a simulation study and a real case study in the quality modeling of a semiconductor manufacturing process. The source code of this work is hosted here: https://github.com/XiaoyuChenUofL/Interpretable-Neural-Network.","PeriodicalId":123526,"journal":{"name":"ACM Transactions on Intelligent Systems and Technology (TIST)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133928436","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"SignDS-FL: Local Differentially Private Federated Learning with Sign-based Dimension Selection","authors":"Xue Jiang, Xuebing Zhou, Jens Grossklags","doi":"10.1145/3517820","DOIUrl":"https://doi.org/10.1145/3517820","url":null,"abstract":"Federated Learning (FL) [31] is a decentralized learning mechanism that has attracted increasing attention due to its achievements in computational efficiency and privacy preservation. However, recent research highlights that the original FL framework may still reveal sensitive information of clients’ local data from the exchanged local updates and the global model parameters. Local Differential Privacy (LDP), as a rigorous definition of privacy, has been applied to Federated Learning to provide formal privacy guarantees and prevent potential privacy leakage. However, previous LDP-FL solutions suffer from considerable utility loss with an increase of model dimensionality. Recent work [29] proposed a two-stage framework that mitigates the dimension-dependency problem by first selecting one “important” dimension for each local update and then perturbing the dimension value to construct the sparse privatized update. However, the framework may still suffer from utility loss because of the insufficient per-stage privacy budget and slow model convergence. In this article, we propose an improved framework, SignDS-FL, which shares the concept of dimension selection with Reference [29], but saves the privacy cost for the value perturbation stage by assigning random sign values to the selected dimensions. Besides using the single-dimension selection algorithms in Reference [29], we propose an Exponential Mechanism-based Multi-Dimension Selection algorithm that further improves model convergence and accuracy. We evaluate the framework on a number of real-world datasets with both simple logistic regression models and deep neural networks. For training logistic regression models on structured datasets, our framework yields only a ( sim ) 1%–2% accuracy loss in comparison to a ( sim ) 5%–15% decrease of accuracy for the baseline methods. For training deep neural networks on image datasets, the accuracy loss of our framework is less than ( 8% ) and at best only ( 2% ) . Extensive experimental results show that our framework significantly outperforms the previous LDP-FL solutions and enjoys an advanced utility-privacy balance.","PeriodicalId":123526,"journal":{"name":"ACM Transactions on Intelligent Systems and Technology (TIST)","volume":"12 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-03-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116773551","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}