Yanshan Xiao , Mengyue Zeng , Bo Liu , Liang Zhao , Xiangjun Kong , Zhifeng Hao
{"title":"Multi-task ordinal regression with task weight discovery","authors":"Yanshan Xiao , Mengyue Zeng , Bo Liu , Liang Zhao , Xiangjun Kong , Zhifeng Hao","doi":"10.1016/j.knosys.2024.112616","DOIUrl":"10.1016/j.knosys.2024.112616","url":null,"abstract":"<div><div>Ordinal regression (OR) deals with the classification problems that the classes are ranked in order. At present, most OR approaches are designed for individual tasks, the research on multi-task OR is limited. These multi-task OR approaches assume that different tasks have the same relatedness and contribute equally to the overall model. However, in practice, different tasks may have distinct relatedness to the overall model. If they are treated equally, the performance of the overall model may be restricted. In this paper, we propose a novel multi-task OR approach with task weight discovery (MORTD). We assign each task a weight that indicates its relatedness to the overall model. Based on the task weights, a maximum margin multi-task OR model is constructed. Then, we adopt a heuristic framework to construct the multi-task OR classifier and update the task weights alternately. In this framework, the dual coordinate descent method is adapted to train the multi-task OR classifier efficiently. In real-world OR applications, the relatedness of multiple tasks may not be exactly the same. The contribution of MORTD is that it can discover the weights of tasks to yield a more precise classification model. Substantial experiments on real-life OR datasets illustrate that compared to the existing multi-task OR methods, MORTD is able to deliver higher classification accuracy and meanwhile needs less training time.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142529374","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jingyu Zhu , Minghua Wan , Guowei Yang , Zhangjing Yang
{"title":"INCOMPLETE multi-view clustering based on low-rank adaptive graph learning","authors":"Jingyu Zhu , Minghua Wan , Guowei Yang , Zhangjing Yang","doi":"10.1016/j.knosys.2024.112562","DOIUrl":"10.1016/j.knosys.2024.112562","url":null,"abstract":"<div><div>The challenge of acquiring complete data has led to substantial progress in incomplete multi-view clustering (IMVC) methods. Because graph structures can be excellent representations of data structure relationships, exceptional performance in handling incomplete data is demonstrated by graph-based methods at present. However, these methods still have their limitations. Most incomplete multi-view algorithms primarily focus on local information, neglecting global information. Therefore, these methods cannot dynamically recover the structural relationships in incomplete data by harnessing potential information from multiple perspectives and overall structural information. In response to the aforementioned concerns, we introduced an IMVC based on low-rank adaptive graph learning (IMVC-LAGL). This method initially constructs an affinity matrix based on the inter-view adjacency relationships. It also utilizes tensor low-rank constraints and consensus representation learning to explore higher-order correlations among different views. Subsequently, it adaptively reconstructs the incomplete graph structure to ultimately obtain a complete affinity relationship. It leads to excellent clustering results by integrating relevant information within views, overall structural information and potential information from multiple perspectives. We conducted experiments comparing our algorithm with eight incomplete multi-view algorithms using five different evaluation metrics. The results show that our algorithm achieves the best clustering results across eight datasets with varying missing rates. Particularly in the BBCSport dataset and YaleB dataset, the clustering accuracy of our algorithm is improved by 19.83 % and 16.41 %, respectively, compared with the second-best algorithm, under a 50 % missing rate.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142529401","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Improving quaternion neural networks with quaternionic activation functions","authors":"Johannes Pöppelbaum, Andreas Schwung","doi":"10.1016/j.knosys.2024.112619","DOIUrl":"10.1016/j.knosys.2024.112619","url":null,"abstract":"<div><div>In this paper, we propose novel quaternion activation functions where we modify either the quaternion magnitude or the phase, as an alternative to the commonly used split activation functions. We define criteria that are relevant for quaternion activation functions, and subsequently we propose our novel activation functions based on this analysis. Instead of applying a known activation function like the ReLU or Tanh on the quaternion elements separately, these activation functions consider the quaternion properties and respect the quaternion space <span><math><mi>H</mi></math></span>. In particular, all quaternion components are utilized to calculate all output components, carrying out the benefit of the Hamilton product in e.g. the quaternion convolution to the activation functions. The proposed activation functions can be incorporated in arbitrary quaternion valued neural networks trained with gradient descent techniques. We further discuss the derivatives of the proposed activation functions where we observe beneficial properties for the activation functions affecting the phase. Specifically, they prove to be sensitive on basically the whole input range, thus improved gradient flow can be expected. We provide an elaborate experimental evaluation of our proposed quaternion activation functions including comparison with the split ReLU and split Tanh on two image classification tasks using the CIFAR-10 and SVHN dataset. There, especially the quaternion activation functions affecting the phase consistently prove to provide better performance.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142529488","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Similarity-driven adversarial testing of neural networks","authors":"Katarzyna Filus, Joanna Domańska","doi":"10.1016/j.knosys.2024.112621","DOIUrl":"10.1016/j.knosys.2024.112621","url":null,"abstract":"<div><div>Although Convolutional Neural Networks (CNNs) are among the most important algorithms of computer vision and the artificial intelligence-based systems, they are vulnerable to adversarial attacks. Such attacks can cause dangerous consequences in real-life deployments. Consequently, testing of the artificial intelligence-based systems from their perspective is crucial to reliably support human prediction and decision-making through computation techniques under varying conditions. While proposing new effective attacks is important for neural network testing, it is also crucial to design effective strategies that can be used to choose target labels for these attacks. That is why, in this paper we propose a novel similarity-driven adversarial testing methodology for target label choosing. Our motivation is that CNNs, similarly to humans, tend to make mistakes mostly among categories they perceive similar. Thus, the effort to make models predict a particular class is not equal for all classes. Motivated by this, we propose to use the most and least similar labels to the ground truth according to different similarity measures to choose the target label for an adversarial attack. They can be treated as best- and worst-case scenarios in practical and transparent testing methodologies. As similarity is one of the key components of human cognition and categorization, the approach presents a shift towards a more human-centered security testing of deep neural networks. The obtained numerical results show the superiority of the proposed methods to the existing strategies in the targeted and the non-targeted testing setups.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142553808","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hai-Long Yang , Yin-Feng Zhou , Jin-Jin Li , Weiping Ding
{"title":"A concept fringe-based concept-cognitive learning method in skill context","authors":"Hai-Long Yang , Yin-Feng Zhou , Jin-Jin Li , Weiping Ding","doi":"10.1016/j.knosys.2024.112618","DOIUrl":"10.1016/j.knosys.2024.112618","url":null,"abstract":"<div><div>Concept-cognitive learning has achieved remarkable results in simulating the learning of concepts. However, the existing concept-cognitive learning models mainly focus on how knowledge is acquired, but ignore the fact that knowledge transfer and knowledge forgetting may occur during the process of learning skills and solving items. This limits the application of concept-cognitive learning in predicting knowledge states and assessing competence states in skill contexts. To overcome this limitation, this paper provides a new concept-cognitive learning method for property-oriented concepts and object-oriented concepts in skill context. Corresponding to the conjunctive model and the disjunctive model, the inner and outer fringes of property-oriented concept and object-oriented concept are first defined, respectively. In this way, items or skills that are easily forgotten and those that are in the zone of proximal development can be found under both models. Furthermore, the Jaccard similarity coefficient is used to diversify the learning outcomes by finding items and skills that are most likely to occur knowledge forgetting or knowledge transfer. Thus, based on the fringes of concepts, the algorithms to learn property-oriented concepts and object-oriented concepts are provided, respectively. Finally, the case study on a real world example and the experimental evaluation on six data sets from UCI demonstrate that the proposed method is of practical significance and effective in terms of running time.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142529405","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Saeid Jafarzadeh Ghoushchi , Abbas Mardani , Luis Martínez
{"title":"Trust number: Trust-based modeling for handling decision-making problems","authors":"Saeid Jafarzadeh Ghoushchi , Abbas Mardani , Luis Martínez","doi":"10.1016/j.knosys.2024.112631","DOIUrl":"10.1016/j.knosys.2024.112631","url":null,"abstract":"<div><div>Fuzzy sets play an effective role in dealing with the uncertainty and ambiguity of input data in real-world decision-making problems. Nevertheless, the effectiveness of fuzzy sets becomes unreliable and even more uncertain when the input data come from untrustworthy sources. Therefore, a new measurement could be considered based on the data's degree of trust to reduce the deviation of unreliable information in fuzzy decision-making problems. The main aim of this study is to introduce a new information modeling called trust numbers (T-numbers), which models variations and deviations associated with triangular fuzzy numbers and their application to decision-making. In addition, it introduces new operations on T-numbers to develop a decision model based on this theory. The performance of this model was analyzed through its implementation in two case studies and by comparing the fuzzy technique for order of Preference by similarity to the ideal solution (F-TOPSIS) and its T-number extension(T-TOPSIS). Results indicate that T-numbers can be applied to classical fuzzy numbers when the available information is uncertain and a degree of distrust exists.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142529486","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"CBRec: A causal way balancing multidimensional attraction effect in POI recommendations","authors":"Bo Liu, Jun Zeng, Junhao Wen, Min Gao, Wei Zhou","doi":"10.1016/j.knosys.2024.112607","DOIUrl":"10.1016/j.knosys.2024.112607","url":null,"abstract":"<div><div>In the next Point-of-Interest recommendation, sparse and uneven location data generate biases, resulting in homogeneous recommendation outcomes that fail to reflect user preferences. Although there are many related unbiased studies, they still exhibit limitations. They lack a unified debiasing paradigm and typically employ different methods to address various biases, resulting in complex and incompatible debiasing models. Additionally, they often overlook the potential advantages of biases, thus harming the quality of location features. To address these challenges, we propose a unified debiasing paradigm by intervening in location attraction to balance the positive and negative effects of bias. By analyzing the structural causal graph, we identify attraction as a feature influenced by bias. By comparing observational results affected by attraction with counterfactual results unaffected by it, we derive a unified debiasing paradigm that eliminates the effects of bias. Additionally, through feature fusion, we embed multidimensional attraction into user features, leveraging the advantages of bias to preserve the quality of location features. Finally, experimental results on five real-world datasets demonstrate that our proposed model outperforms recent sequential recommendation models.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142446717","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Local density based on weighted K-nearest neighbors for density peaks clustering","authors":"Sifan Ding , Min Li , Tianyi Huang , William Zhu","doi":"10.1016/j.knosys.2024.112609","DOIUrl":"10.1016/j.knosys.2024.112609","url":null,"abstract":"<div><div>Density peaks clustering (DPC), a traditional density-based clustering algorithm, has received considerable attention in recent years. DPC identifies clusters by designating density peaks, defined by local density, as cluster centers. However, DPC and its variants often struggle to identify high-density peaks, particularly in datasets with arbitrarily complex shapes. To address this issue, we propose a novel local density measure based on weighted K-nearest neighbors (KNN). First, we construct a new similarity measure, termed the constrained kernel rank-order distance, to determine the KNNs of each point. Next, we develop the concept of weighted KNNs by assigning a weight to each point, representing the probability of it becoming a KNN to other points. Subsequently, we redefine the local density based on the weighted KNN. Finally, we integrate this new local density measure into the DPC framework. Experiments demonstrate that the proposed algorithm outperforms existing DPC algorithms in terms of effectiveness. The source code can be downloaded from <span><span>https://github.com/Gedanke/dpcCode</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142529376","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"GA-SmaAt-GNet: Generative adversarial small attention GNet for extreme precipitation nowcasting","authors":"Eloy Reulen, Jie Shi, Siamak Mehrkanoon","doi":"10.1016/j.knosys.2024.112612","DOIUrl":"10.1016/j.knosys.2024.112612","url":null,"abstract":"<div><div>In recent years, data-driven modeling approaches have gained significant attention across various meteorological applications, particularly in weather forecasting. However, these methods often face challenges in handling extreme weather conditions. In response, we present the GA-SmaAt-GNet model, a novel generative adversarial framework for extreme precipitation nowcasting. This model features a unique SmaAt-GNet generator, an extension of the successful SmaAt-UNet architecture, capable of integrating precipitation masks (binarized precipitation maps) to enhance predictive accuracy. Additionally, GA-SmaAt-GNet incorporates an attention-augmented discriminator inspired by the Pix2Pix architecture. This innovative framework paves the way for generative precipitation nowcasting using multiple data sources. We evaluate the performance of SmaAt-GNet and GA-SmaAt-GNet using real-life precipitation data from The Netherlands, revealing notable improvements in overall performance and for extreme precipitation events compared to other models. Specifically, our proposed architecture demonstrates its main performance gain in summer and autumn, when precipitation intensity is typically at its peak. Furthermore, we conduct uncertainty analysis on the GA-SmaAt-GNet model and the precipitation dataset, providing insights into its predictive capabilities. Finally, we employ Grad-CAM to offer visual explanations of our model’s predictions, generating activation heatmaps that highlight areas of input activation throughout the network.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142446715","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Minjie Feng , Haidong Shao , Minghui Shao , Yiming Xiao , Jie Wang , Bin Liu
{"title":"Utilizing Bayesian generalization network for reliable fault diagnosis of machinery with limited data","authors":"Minjie Feng , Haidong Shao , Minghui Shao , Yiming Xiao , Jie Wang , Bin Liu","doi":"10.1016/j.knosys.2024.112628","DOIUrl":"10.1016/j.knosys.2024.112628","url":null,"abstract":"<div><div>To address the issues of overfitting, domain generalization challenges, and lack of credibility brought by limited data samples in mechanical fault diagnosis in practical engineering, this paper proposes a reliable Bayesian generalization network (BGNet). A Bayesian convolutional layer is constructed based on variational inference, treating all parameters in the convolutional layer as random variables. This approach makes a single model function similar to an ensemble of an infinite number of models, and thus enhancing the model's capability of overfitting resistance and domain generalization. The parameters of the variational distribution are updated to approximate the posterior distribution by local reparametrization and Monte Carlo sampling to optimize the evidence lower bound (ELBO) loss. Confidence information is extracted from the model results and, uncertainty estimation and decomposition schemes are designed to provide interpretability. The proposed method is applied to analyze the experimental data of bearing and gearbox faults. The results show that in a multi-source domain scenario with limited samples, the proposed method demonstrates high diagnostic accuracy, effectively describes the relationship between domain variability and uncertainty, and significantly outperforms several benchmark and state-of-the-art models.</div></div>","PeriodicalId":49939,"journal":{"name":"Knowledge-Based Systems","volume":null,"pages":null},"PeriodicalIF":7.2,"publicationDate":"2024-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142442339","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}