{"title":"Machine Unlearning Through Fine-Grained Model Parameters Perturbation","authors":"Zhiwei Zuo;Zhuo Tang;Kenli Li;Anwitaman Datta","doi":"10.1109/TKDE.2025.3528551","DOIUrl":"https://doi.org/10.1109/TKDE.2025.3528551","url":null,"abstract":"Machine unlearning involves retracting data records and reducing their influence on trained models, aiding user privacy protection, at a significant computational cost potentially. Weight perturbation-based unlearning is common but typically modifies parameters globally. We propose fine-grained Top-K and Random-k parameters perturbed inexact machine unlearning that address the privacy needs while keeping the computational costs tractable. However, commonly used training data are independent and identically distributed, for inexact machine unlearning, current metrics are inadequate in quantifying unlearning degree that occurs after unlearning. To address this quantification issue, we introduce SPD-GAN, which subtly perturbs data distribution targeted for unlearning. Then, we evaluate unlearning degree by measuring the performance difference of the models on the perturbed unlearning data before and after unlearning. Furthermore, to demonstrate efficacy, we tackle the challenge of evaluating machine unlearning by assessing model generalization across unlearning and remaining data. To better assess the unlearning effect and model generalization, we propose novel metrics, namely, the forgetting rate and memory retention rate. By implementing these innovative techniques and metrics, we achieve computationally efficacious privacy protection in machine learning applications without significant sacrifice of model performance. A by-product of our work is a novel method for evaluating and quantifying unlearning degree.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 4","pages":"1975-1988"},"PeriodicalIF":8.9,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143570561","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Adversarial Conservative Alternating Q-Learning for Credit Card Debt Collection","authors":"Wenhui Liu;Jiapeng Zhu;Lyu Ni;Jingyu Bi;Zhijian Wu;Jiajie Long;Mengyao Gao;Dingjiang Huang;Shuigeng Zhou","doi":"10.1109/TKDE.2025.3528219","DOIUrl":"https://doi.org/10.1109/TKDE.2025.3528219","url":null,"abstract":"Debt collection is utilized for risk control after credit card delinquency. The existing rule-based method tends to be myopic and non-adaptive due to the delayed feedback. Reinforcement learning (RL) has an inherent advantage in dealing with such task and can learn policies end-to-end. However, employing RL here remains difficult because of different interaction processes from standard RL and the notorious problem of optimistic estimations in the offline setting. To tackle these challenges, we first propose an Alternating Q-Learning (AQL) framework to adapt debt collection processes to comparable procedures in RL. Based on AQL, we further develop an Adversarial Conservative Alternating Q-Learning (ACAQL) to address the issue of overoptimistic estimations. Specifically, adversarial conservative value regularization is proposed to balance optimism and conservatism on Q-values of out-of-distribution actions. Furthermore, ACAQL utilizes the counterfactual action stitching to mitigate the overestimation by enhancing behavior data. Finally, we evaluate ACAQL on a real-world dataset created from Bank of Shanghai. Offline experimental results show that our approach outperforms state-of-the-art methods and effectively alleviates the optimistic estimation issue. Moreover, we conduct online A/B tests on the bank, and ACAQL achieves at least a <italic>6%</i> improvement of the debt recovery rate, which yields tangible economic benefits.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 4","pages":"1542-1555"},"PeriodicalIF":8.9,"publicationDate":"2025-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143570828","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"FedDict: Towards Practical Federated Dictionary-Based Time Series Classification","authors":"Zhiyu Liang;Zheng Liang;Hongzhi Wang;Bo Zheng","doi":"10.1109/TKDE.2025.3528023","DOIUrl":"https://doi.org/10.1109/TKDE.2025.3528023","url":null,"abstract":"The dictionary-based approach is one of the most representative types of time series classification (TSC) algorithm due to its high accuracy, efficiency, and good interpretability. However, existing studies focus on the centralized scenario where data from multiple sources are gathered. Considering that in many practical applications, data owners are reluctant to share their data due to privacy concerns, we study an unexplored problem involving collaboratively building the dictionary-based model over the data owners without disclosing their private data (i.e., in the federated scenario). We propose FedDict, a novel dictionary-based TSC approach customized for the federated setting to benefit from the advantages of the centralized algorithms. To further improve the performance and practicality, we propose a novel federated optimization algorithm for training logistic regression classifiers using dictionary features. The algorithm does not rely on any secure broker and is more accurate and efficient than existing solutions without hyper-parameter tuning. We also propose two contract algorithms for federated dictionary building, such that the user can flexibly balance the running time and the TSC performance through a pre-defined time limit. Extensive experiments on a total of 117 highly heterogeneous datasets validate the effectiveness of our methods and the superiority over existing solutions.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 4","pages":"1785-1803"},"PeriodicalIF":8.9,"publicationDate":"2025-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143570655","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jiayun Zhang;Xinyang Zhang;Dezhi Hong;Rajesh K. Gupta;Jingbo Shang
{"title":"Contextual Inference From Sparse Shopping Transactions Based on Motif Patterns","authors":"Jiayun Zhang;Xinyang Zhang;Dezhi Hong;Rajesh K. Gupta;Jingbo Shang","doi":"10.1109/TKDE.2024.3452638","DOIUrl":"https://doi.org/10.1109/TKDE.2024.3452638","url":null,"abstract":"Inferring contextual information such as demographics from historical transactions is valuable to public agencies and businesses. Existing methods are data-hungry and do not work well when the available records of transactions are sparse. We consider here specifically inference of demographic information using limited historical grocery transactions from a few random trips that a typical business or public service organization may see. We propose a novel method called \u0000<sc>DemoMotif</small>\u0000 to build a network model from heterogeneous data and identify subgraph patterns (i.e., motifs) that enable us to infer demographic attributes. We then design a novel motif context selection algorithm to find specific node combinations significant to certain demographic groups. Finally, we learn representations of households using these selected motif instances as context, and employ a standard classifier (e.g., SVM) for inference. For evaluation purposes, we use three real-world consumer datasets, spanning different regions and time periods in the U.S. We evaluate the framework for predicting three attributes: ethnicity, seniority of household heads, and presence of children. Extensive experiments and case studies demonstrate that \u0000<sc>DemoMotif</small>\u0000 is capable of inferring household demographics using only a small number (e.g., fewer than 10) of random grocery trips, significantly outperforming the state-of-the-art.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 2","pages":"572-583"},"PeriodicalIF":8.9,"publicationDate":"2025-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142940718","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Letian Gong;Shengnan Guo;Yan Lin;Yichen Liu;Erwen Zheng;Yiwei Shuang;Youfang Lin;Jilin Hu;Huaiyu Wan
{"title":"STCDM: Spatio-Temporal Contrastive Diffusion Model for Check-In Sequence Generation","authors":"Letian Gong;Shengnan Guo;Yan Lin;Yichen Liu;Erwen Zheng;Yiwei Shuang;Youfang Lin;Jilin Hu;Huaiyu Wan","doi":"10.1109/TKDE.2025.3525718","DOIUrl":"https://doi.org/10.1109/TKDE.2025.3525718","url":null,"abstract":"Analyzing and comprehending check-in sequences is crucial for various applications in smart cities. However, publicly available check-in datasets are often limited in scale due to privacy concerns. This poses a significant obstacle to academic research and downstream applications. Thus, it is urgent to generate realistic check-in datasets. The denoising diffusion probabilistic model (DDPM) as one of the most capable generation methods is a good choice to achieve this goal. However, generating check-in sequences using DDPM is not an easy feat. The difficulties lie in handling check-in sequences of variable lengths and capturing the correlation from check-in sequences’ distinct characteristics. This paper addresses the challenges by proposing a Spatio-Temporal Contrastive Diffusion Model (STCDM). This model introduces a novel spatio-temporal lossless encoding method that effectively encodes check-in sequences into a suitable format with equal length. Furthermore, we capture the spatio-temporal correlations with two disentangled diffusion modules to reduce the impact of the difference between spatial and temporal characteristics. Finally, we incorporate contrastive learning to enhance the relationship between diffusion modules. We generate four realistic datasets in different scenarios using STCDM and design four metrics for comparison. Experiments demonstrate that our generated datasets are more realistic and free of privacy leakage.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 4","pages":"2141-2154"},"PeriodicalIF":8.9,"publicationDate":"2025-01-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143570791","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Preference-Consistent Knowledge Distillation for Recommender System","authors":"Zhangchi Zhu;Wei Zhang","doi":"10.1109/TKDE.2025.3526420","DOIUrl":"https://doi.org/10.1109/TKDE.2025.3526420","url":null,"abstract":"Feature-based knowledge distillation has been applied to compress modern recommendation models, usually with projectors that align student (small) recommendation models’ dimensions with teacher dimensions. However, existing studies have only focused on making the projected features (i.e., student features after projectors) similar to teacher features, overlooking investigating whether the user preference can be transferred to student features (i.e., student features before projectors) in this manner. In this paper, we find that due to the lack of restrictions on projectors, the process of transferring user preferences will likely be interfered with. We refer to this phenomenon as preference inconsistency. It greatly wastes the power of feature-based knowledge distillation. To mitigate preference inconsistency, we propose PCKD, which consists of two regularization terms for projectors. We also propose a hybrid method that combines the two regularization terms. We focus on items with high preference scores and significantly mitigate preference inconsistency, improving the performance of feature-based knowledge distillation. Extensive experiments on three public datasets and three backbones demonstrate the effectiveness of PCKD.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 4","pages":"2071-2084"},"PeriodicalIF":8.9,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143570792","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"ADMH-ER: Adaptive Denoising Multi-Modal Hybrid for Entity Resolution","authors":"Qian Zhou;Wei Chen;Li Zhang;An Liu;Junhua Fang;Lei Zhao","doi":"10.1109/TKDE.2025.3526623","DOIUrl":"https://doi.org/10.1109/TKDE.2025.3526623","url":null,"abstract":"Multi-Modal Knowledge Graphs (MMKGs), comprising relational triples and related multi-modal data (e.g., text and images), usually suffer from the problems of low coverage and incompleteness. To mitigate this, existing studies introduce a fundamental MMKG fusion task, i.e., Multi-Modal Entity Alignment (MMEA) that identifies equivalent entities across multiple MMKGs. Despite MMEA’s significant advancements, effectively integrating MMKGs remains challenging, mainly stemming from two core limitations: 1) entity ambiguity, where real-world entities across different MMKGs may possess multiple corresponding counterparts or alternative identities; and 2) severe noise within multi-modal data. To tackle these limitations, a new task MMER (Multi-Modal Entity Resolution), which expands the scope of MMEA to encompass entity ambiguity, is introduced. To tackle this task effectively, we develop a novel model ADMH-ER (Adaptive Denoising Multi-modal Hybrid for Entity Resolution) that incorporates several crucial modules: 1) multi-modal knowledge encoders, which are crafted to obtain entity representations based on multi-modal data sources; 2) an adaptive denoising multi-modal hybrid module that is designed to tackle challenges including noise interference, multi-modal heterogeneity, and semantic irrelevance across modalities; and 3) a hierarchical multi-objective learning strategy, which is proposed to ensure diverse convergence capabilities among different learning objectives. Experimental results demonstrate that ADMH-ER outperforms state-of-the-art methods.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 3","pages":"1049-1063"},"PeriodicalIF":8.9,"publicationDate":"2025-01-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143106833","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"MagicNet: Memory-Aware Graph Interactive Causal Network for Multivariate Stock Price Movement Prediction","authors":"Di Luo;Shuqi Li;Weiheng Liao;Rui Yan","doi":"10.1109/TKDE.2025.3527480","DOIUrl":"https://doi.org/10.1109/TKDE.2025.3527480","url":null,"abstract":"Quantitative trading is a prominent field that employs time series analysis today, attracting researchers who apply machine intelligence to real-world issues like stock price movement prediction. In recent literature, various types of auxiliary data have been integrated alongside stock prices to improve prediction accuracy, such as textual news and correlational information. However, they typically rely on directly related documents or symmetric price correlations to make predictions for a particular stock (we refer to as “self-influence”). In this paper, we propose a Memory-Aware Graph Interactive Causal Network (MagicNet) that considers both temporal and spatial dependencies in financial documents and introduces causality-based correlations between multivariate stocks in a hierarchical fashion. MagicNet involves a text memory slot for each stock to retain the most influential texts over time and contains a dynamic interaction graph based on causal relationships to aggregate interactive influences asymmetrically. We believe that MagicNet leverages influential texts across stocks and explores their interrelationships through a logical structure, improving predictions on multiple stocks (we refer to as “interactive-influence”). The effectiveness of MagicNet is demonstrated through experiments on three real-world datasets, where MagicNet outperforms existing state-of-the-art models, offering an intuitive framework for understanding how texts and correlations affect future stock prices.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 4","pages":"1989-2000"},"PeriodicalIF":8.9,"publicationDate":"2025-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143570803","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zihan Zhang;Xiao Ding;Xia Liang;Yusheng Zhou;Bing Qin;Ting Liu
{"title":"Brain and Cognitive Science Inspired Deep Learning: A Comprehensive Survey","authors":"Zihan Zhang;Xiao Ding;Xia Liang;Yusheng Zhou;Bing Qin;Ting Liu","doi":"10.1109/TKDE.2025.3527551","DOIUrl":"https://doi.org/10.1109/TKDE.2025.3527551","url":null,"abstract":"Deep learning (DL) is increasingly viewed as a foundational methodology for advancing Artificial Intelligence (AI). However, its interpretability remains limited, and it often underperforms in certain fields due to its lack of human-like characteristics. Consequently, leveraging insights from Brain and Cognitive Science (BCS) to understand and advance DL has become a focal point for researchers in the DL community. However, BCS is a diverse discipline where existing studies often concentrate on cognitive theories within their respective domains. These theories are typically grounded in certain assumptions, complicating comparisons between different approaches. Therefore, this review is intended to provide a comprehensive landscape of more than 300 papers on the intersection of DL and BCS grounded in DL community. Unlike previous reviews that based on sub-disciplines of Cognitive Science, this article aims to establish a unified framework encompassing all aspects of DL inspired by BCS, offering insights into the symbiotic relationship between DL and BCS. Additionally, we present a forward-looking perspective on future research directions, with the intention of inspiring further advancements in AI research.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 4","pages":"1650-1671"},"PeriodicalIF":8.9,"publicationDate":"2025-01-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143570615","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"DASKT: A Dynamic Affect Simulation Method for Knowledge Tracing","authors":"Xinjie Sun;Kai Zhang;Qi Liu;Shuanghong Shen;Fei Wang;Yuxiang Guo;Enhong Chen","doi":"10.1109/TKDE.2025.3526584","DOIUrl":"https://doi.org/10.1109/TKDE.2025.3526584","url":null,"abstract":"Knowledge Tracing (KT) predicts future performance by modeling students’ historical interactions, and understanding students’ affective states can enhance the effectiveness of KT, thereby improving the quality of education. Although traditional KT values students’ cognition and learning behaviors, efficient evaluation of students’ affective states and their application in KT still require further exploration due to the non-affect-oriented nature of the data and budget constraints. To address this issue, we propose a computation-driven approach, <bold>D</b>ynamic <bold>A</b>ffect <bold>S</b>imulation <bold>K</b>nowledge <bold>T</b>racing (DASKT), to explore the impact of various student affective states (such as frustration, concentration, boredom, and confusion) on their knowledge states. In this model, we first extract affective factors from students’ non-affect-oriented behavioral data, then use clustering and spatiotemporal sequence modeling to accurately simulate students’ dynamic affect changes when dealing with different problems. Subsequently, we incorporate affect with time-series analysis to improve the model's ability to infer knowledge states over time and space. Extensive experimental results on two public real-world educational datasets show that DASKT can achieve more reasonable knowledge states under the effect of students’ affective states. Moreover, DASKT outperforms the most advanced KT methods in predicting student performance. Our research highlights a promising avenue for future KT studies, focusing on achieving high interpretability and accuracy.","PeriodicalId":13496,"journal":{"name":"IEEE Transactions on Knowledge and Data Engineering","volume":"37 4","pages":"1714-1727"},"PeriodicalIF":8.9,"publicationDate":"2025-01-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143570594","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}