Knowledge-Based Systems最新文献

Adaptive class token knowledge distillation for efficient vision transformer 自适应类标记知识提炼，实现高效视觉转换器

IF 7.2 1区计算机科学

Knowledge-Based Systems Pub Date : 2024-09-19 DOI: 10.1016/j.knosys.2024.112531

{"title":"Adaptive class token knowledge distillation for efficient vision transformer","authors":"","doi":"10.1016/j.knosys.2024.112531","DOIUrl":"10.1016/j.knosys.2024.112531","url":null,"abstract":"<div><p>The Vision Transformer (ViT) outperforms Convolutional Neural Networks (CNNs) but at the cost of significantly higher computational demands. Knowledge Distillation (KD) has shown promise in compressing complex networks by transferring knowledge from a large pre-trained model to a smaller one. However, current KD methods for ViT often rely on CNNs as teachers or neglect the importance of class token ([CLS]) information, resulting in ineffective distillation of ViT’s unique knowledge. In this paper, we propose Adaptive Class token Knowledge Distillation ([CLS]-KD), which fully exploits information from the class token and patches in ViT. For class embedding (CLS) distillation, the intermediate CLS of the student model is aligned with the corresponding CLS of the teacher model through a projector. Furthermore, we introduce CLS-patch attention map distillation, where an attention map between the CLS and patch embeddings is generated and matched at each layer. This empowers the student model to learn how to dynamically extract patch embedding information into the CLS under teacher guidance. Finally, we propose Adaptive Layer-wise Distillation (ALD) to mitigate the imbalance in distillation effects varying with the depth of layers. This method assigns greater weight to the losses in layers where the training discrepancies between the teacher and student models are larger during distillation. Through these strategies, [CLS]-KD consistently surpasses existing state-of-the-art methods on the ImageNet-1K dataset across various teacher–student configurations. Furthermore, the proposed method demonstrates its generalization capability through transfer learning experiments on the CIFAR-10, CIFAR-100, and CALTECH-256 datasets.</p></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142272580","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

The performance of priority rules for the decentralized resource-constrained multi-project scheduling 分散式资源受限多项目调度优先规则的性能

IF 7.2 1区计算机科学

Knowledge-Based Systems Pub Date : 2024-09-18 DOI: 10.1016/j.knosys.2024.112530

{"title":"The performance of priority rules for the decentralized resource-constrained multi-project scheduling","authors":"","doi":"10.1016/j.knosys.2024.112530","DOIUrl":"10.1016/j.knosys.2024.112530","url":null,"abstract":"<div><p>Decentralized resource-constrained multi-project scheduling (DRCMPSP) is becoming increasingly common in construction, supply chains, and many other industrial disciplines. DRCMPSP faces difficult decisions in resolving resource conflicts to generate a baseline schedule to optimize global objectives. We propose an agent-based approach to address the DRCMPSP based on two global objectives: average project delay and total project delay. A heuristic based on the priority rule (PR) is developed to coordinate the global resource allocation. A comprehensive analysis of 30 PRs was conducted on 16,000 portfolios containing 48,000 projects . We confirmed that using the same PR to allocate global resources on all occasions often results in unnecessarily poor performance. The best PR depends on project and portfolio characteristics such as serial/parallel indicators, global resource distribution, and tightness. Moreover, the best PR differs from various perspectives (e.g., projects and portfolios). We summarized our results in three decision tables and further distilled these results for practical use, which only provide a rough estimate of the project and portfolio characteristics.</p></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142272581","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

DCTracker: Rethinking MOT in soccer events under dual views via cascade association DCTracker：通过级联重新思考双重视角下足球赛事中的 MOT

IF 7.2 1区计算机科学

Knowledge-Based Systems Pub Date : 2024-09-18 DOI: 10.1016/j.knosys.2024.112528

{"title":"DCTracker: Rethinking MOT in soccer events under dual views via cascade association","authors":"","doi":"10.1016/j.knosys.2024.112528","DOIUrl":"10.1016/j.knosys.2024.112528","url":null,"abstract":"<div><p>Multi-Object Tracking (MOT) holds significant potential for enhancing the analysis of sporting events. Traditional MOT models are primarily designed for pedestrian-centric scenarios with static cameras and linear motion patterns. However, the dynamic environment of sports presents unique challenges: (i) significant camera movements and dynamic focal length adjustments cause abrupt changes in player positions across frames; (ii) player trajectories are nonlinear and influenced by game dynamics, resulting in complex, rapid movements complicated by erratic camera motion; and (iii) issues like image blurring, occlusion, and similar player appearances challenge visual identification robustness. These factors create substantial obstacles for standard tracking algorithms. To address these challenges, we introduce DCTracker, a specialized MOT system for robust performance in soccer matches. Our approach enhances the conventional Kalman filter by integrating a bird’s-eye view via homography and inter-frame registration for the broadcast view, termed the dual-view Kalman filter (DVKF). This method leverages context from both perspectives to enrich the estimation model with multi-state vectors for each object, mitigating the impact of camera motion and nonlinear trajectories. We also introduce the cascade selection module (CSM), which optimizes the strengths of each perspective by dynamically adjusting their influence using spatial topological relationships among players. The CSM creates an adaptive cost matrix that effectively manages visual issues from blurring and occlusion. The efficacy of our method is demonstrated through state-of-the-art performance on the SoccerNet-Tracking test set and the SportsMOT-soccer validation split, highlighting its robustness across diverse venues and challenging player trajectories.</p></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142272577","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Causally aware reinforcement learning agents for autonomous cyber defence 用于自主网络防御的因果意识强化学习代理

IF 7.2 1区计算机科学

Knowledge-Based Systems Pub Date : 2024-09-17 DOI: 10.1016/j.knosys.2024.112521

{"title":"Causally aware reinforcement learning agents for autonomous cyber defence","authors":"","doi":"10.1016/j.knosys.2024.112521","DOIUrl":"10.1016/j.knosys.2024.112521","url":null,"abstract":"<div><p>Artificial Intelligence (AI) is seen as a disruptive solution to the ever increasing security threats on network infrastructures. To automate the process of defending networked environments from such threats, approaches such as Reinforcement Learning (RL) have been used to train agents in cyber adversarial games. One primary challenge is how contextual information could be integrated into RL models to create agents which adapt their behaviour to adversarial posture. Two desirable characteristics identified for such models are that they should be interpretable and causal.</p><p>To address this challenge, we propose an approach through the integration of a causal rewards model with a modified Proximal Policy Optimisation (PPO) agent in Meta’s MBRL-Lib framework. Our RL agents are trained and evaluated against a range of cyber-relevant scenarios in the Dstl YAWNING-TITAN (YT) environment. We have constructed and experimented with two types of reward functions to facilitate the agent’s learning process. Evaluation metrics include, among others, games won by the defence agent (blue wins), episode length, healthy nodes and isolated nodes.</p><p>Results show that, over all scenarios, our causally aware agent achieves better performance than causally-blind state-of-the-art benchmarks in these scenarios for the above evaluation metrics. In particular, with our proposed High Value Target (HVT) rewards function, which aims not to disrupt HVT nodes, the number of isolated nodes is improved by 17% and 18% against the model-free and Neural Network (NN) model-based agents across all scenarios. More importantly, the overall performance improvement for the blue wins metric exceeded that of model-free and NN model-based agents by 40% and 17%, respectively, across all scenarios.</p></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142239368","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Aiding decision makers in articulating a preference closeness model through compensatory fuzzy logic for many-objective optimization problems 针对多目标优化问题，通过补偿模糊逻辑帮助决策者阐明偏好接近模型

IF 7.2 1区计算机科学

Knowledge-Based Systems Pub Date : 2024-09-16 DOI: 10.1016/j.knosys.2024.112524

{"title":"Aiding decision makers in articulating a preference closeness model through compensatory fuzzy logic for many-objective optimization problems","authors":"","doi":"10.1016/j.knosys.2024.112524","DOIUrl":"10.1016/j.knosys.2024.112524","url":null,"abstract":"<div><p>One of the main challenges in applying preference-based approaches to many-objective optimization problems is that decision makers (DMs) initially have only a vague notion of the solution they want and can obtain. In this paper, we propose an interactive approach that aids DMs in articulating a preference model in a progressive way. The quality of a solution is determined in terms of its “preference closeness” to an aspiration point, which is a subjective concept that can be outlined by the DM. Our proposal is based on compensatory fuzzy logic, which allows for the construction of predicates that are expressed in language that is close to natural. One main advantage is that the model can be optimized via metaheuristics, and we utilize an ant colony optimization algorithm for this. Our model complies with the principles of hybrid augmented intelligence, not only because the algorithm is enriched with knowledge from the DM, but also because the DM also learns the concept of “preference closeness” throughout the process. The proposed model is validated on benchmarks with five and 10 objective functions, and is compared with two state-of-the-art algorithms. Our approach allows for better convergence to the best compromise solutions. The advantages of our approach are supported by statistical tests of the results.</p></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142272578","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A global contextual enhanced structural-aware transformer for sequential recommendation 用于顺序推荐的全局上下文增强型结构感知转换器

IF 7.2 1区计算机科学

Knowledge-Based Systems Pub Date : 2024-09-16 DOI: 10.1016/j.knosys.2024.112515

{"title":"A global contextual enhanced structural-aware transformer for sequential recommendation","authors":"","doi":"10.1016/j.knosys.2024.112515","DOIUrl":"10.1016/j.knosys.2024.112515","url":null,"abstract":"<div><p>Sequential recommendation (SR) has become a research hotspot recently. In our research, we observe that most existing SR models only leverage each user’s own interaction sequence to make recommendation. We argue that leveraging global contextual information across different interaction sequences could enrich item representations and thereby improve recommendation performance. To achieve this, we formulate a global graph from different sequences, providing global contextual information for each sequence. Specifically, we propose to conduct graph contrastive learning on a subgraph sampled from the global graph and a local sequence graph built from each sequence to augment item representations within each sequence. At the same time, we observe that structural dependencies, referring to relationships between items based on the graphic structure, can be extracted from the constructed global graph. Capturing structural dependencies between items may enrich the item representations. To leverage structural dependencies, we propose a new attention mechanism referred to as the Jaccard attention. While prevalent Transformer-based SR models capture semantic dependencies, referring to relationships between items based on item embeddings, in a sequence through self-attention. Therefore, it is beneficial to capture both semantic and structural dependencies between items in a sequence to further enrich item representations. Specifically, we employ two sequence encoders based on the self-attention and the proposed Jaccard attention to capture semantic and structural dependencies between items in a sequence, respectively. Overall, we propose a Global Contextual enhanced Structural-aware Transformer (GC-ST) for SR. Extensive experiments carried out on three widely used datasets demonstrate the effectiveness of GC-ST.</p></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142255927","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

AGS: Transferable adversarial attack for person re-identification by adaptive gradient similarity attack AGS：通过自适应梯度相似性攻击进行人物再识别的可转移对抗攻击

IF 7.2 1区计算机科学

Knowledge-Based Systems Pub Date : 2024-09-13 DOI: 10.1016/j.knosys.2024.112506

{"title":"AGS: Transferable adversarial attack for person re-identification by adaptive gradient similarity attack","authors":"","doi":"10.1016/j.knosys.2024.112506","DOIUrl":"10.1016/j.knosys.2024.112506","url":null,"abstract":"<div><p>Person re-identification (Re-ID) has achieved tremendous success in the fields of computer vision and security. However, Re-ID models are susceptible to adversarial examples, which are crafted by introducing imperceptible perturbations to benign person images. These adversarial examples often display high success rates in white-box settings but their transferability to black-box settings is relatively low. To improve the transferability of adversarial examples, this paper proposes a novel approach called the adaptive gradient similarity attack (AGS), which encompasses two essential components: gradient similarity and enhanced second moment. Specifically, a gradient similarity modulation is established to better harness the information in the neighborhood of the adjacent input, which can adaptively correct the update direction. Additionally, this paper formulates an enhanced second moment to adjust the update step at each iteration to address the issue of poor transferability. Extensive experiments confirm that AGS achieves the best performance compared with the state-of-the-art gradient-based attacks. Moreover, AGS is a versatile approach that can be integrated with existing input transformation attack techniques. Code is available at <span><span>https://github.com/ZezeTao/similar_Attack4</span><svg><path></path></svg></span>.</p></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142272579","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

LiFSO-Net: A lightweight feature screening optimization network for complex-scale flat metal defect detection LiFSO-Net：用于复杂尺度平面金属缺陷检测的轻量级特征筛选优化网络

IF 7.2 1区计算机科学

Knowledge-Based Systems Pub Date : 2024-09-13 DOI: 10.1016/j.knosys.2024.112520

{"title":"LiFSO-Net: A lightweight feature screening optimization network for complex-scale flat metal defect detection","authors":"","doi":"10.1016/j.knosys.2024.112520","DOIUrl":"10.1016/j.knosys.2024.112520","url":null,"abstract":"<div><p>Defect recognition of flat metals is paramount for ensuring quality control during the production process. However, the diverse origins of metal surface damage, ranging from mechanical impacts to chemical corrosion, and the resulting varied morphology and scale of surface defects, particularly numerous microdefects and elongated defects with high aspect ratios, complicate defect recognition. Existing methods fail to select the most beneficial features during extraction and commonly lose critical feature information during gradient sampling. To overcome these challenges, we propose a lightweight network to optimize feature screening for defect recognition. First, we propose a deformable context<strong>–</strong>guided block that employs deformable convolution to dynamically adapt the perception of the spatial context, providing precise guidance of relevant semantic information in complex surface textures. Second, we develop a content<strong>-</strong>aware feature compression block that implements adaptive weighting of features, which significantly reduces information loss during the downsampling stage. Finally, we introduce an intra-scale feature interaction transformer block, which optimizes high-order semantic features to enhance the accuracy and reliability of defect detection. Experimental validation on the NEU-DET, APS-DET, and GC10-DET datasets demonstrated significant improvements in the detection accuracy and parameter efficiency, confirming the proposed method's robust generalizability.</p></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142239370","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Cross-domain recommender system with embedding- and mapping-based knowledge correlation 基于嵌入和映射知识关联的跨域推荐系统

IF 7.2 1区计算机科学

Knowledge-Based Systems Pub Date : 2024-09-13 DOI: 10.1016/j.knosys.2024.112514

{"title":"Cross-domain recommender system with embedding- and mapping-based knowledge correlation","authors":"","doi":"10.1016/j.knosys.2024.112514","DOIUrl":"10.1016/j.knosys.2024.112514","url":null,"abstract":"<div><p>A knowledge transfer-based cross-domain recommender system is currently a research hotspot. Existing research has reached a high level of maturity in mining potential knowledge and establishing transfer mechanisms. However, most of them ignore the impact of the dissimilarity of potential knowledge on the transfer performance. Herein, a cross-domain recommender system based on knowledge correlation-induced the embedding and mapping approach is proposed, denoted by KCEM-CDRS. First, we propose a knowledge correlation measure, which captures the consistency of knowledge between the target and source domains to build the bridge for knowledge transfer. Second, we construct a joint matrix triple factorization model to solve the data sparsity in the target domain while introducing graph regularization to solve the problem of negative knowledge transfer. Results of extensive experiments on real Amazon metadata indicate that compared with three existing cross-domain recommendation methods, KCEM-CDRS shows performance improvements of 0.05–9.55 % and 0.02–2.63 % on mean absolute error and root mean square error, respectively. Additionally, the results of the ablation experiments indicate that consideration of the knowledge correlation between domains is beneficial for knowledge transfer when the density of the source domain is rich.</p></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142255928","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A reliable Bayesian regularization neural network approach to solve the global stability of infectious disease model 解决传染病模型全局稳定性的可靠贝叶斯正则化神经网络方法

IF 7.2 1区计算机科学

Knowledge-Based Systems Pub Date : 2024-09-13 DOI: 10.1016/j.knosys.2024.112481

{"title":"A reliable Bayesian regularization neural network approach to solve the global stability of infectious disease model","authors":"","doi":"10.1016/j.knosys.2024.112481","DOIUrl":"10.1016/j.knosys.2024.112481","url":null,"abstract":"<div><p>The purpose of this study is to perform the numerical results of the global stability of infectious disease mathematical model by using the stochastic computing scheme. The design of proposed solver is presented by one of the efficient and reliable schemes named as Bayesian regularization neural network (BRNN). The global stability of infectious disease mathematical nonlinear model is categorized into susceptible, infected, recovered and vaccinated. The construction of dataset is performed through the Runge-Kutta scheme in order to lessen the mean square error (MSE) by dividing the statics as training 74 %, while 13 % for both testing and endorsement. The proposed stochastic process contains log-sigmoid merit function, twenty neurons and optimization through RBNN for the numerical solutions of the global stability of infectious disease mathematical system. The best training values for each model's case are performed around 10<sup>–11</sup>. The scheme's correctness is performed by the matching of the results and the minor calculated absolute error performances. Moreover, the regression, state transmission, error histogram and MSE indicate the trustworthiness of the designed solver.</p></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-09-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142239371","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0