Neural NetworksPub Date : 2024-11-23DOI: 10.1016/j.neunet.2024.106914
Jie Wu , Jiquan Ma , Heran Xi , Jinbao Li , Jinghua Zhu
{"title":"Multi-scale graph harmonies: Unleashing U-Net’s potential for medical image segmentation through contrastive learning","authors":"Jie Wu , Jiquan Ma , Heran Xi , Jinbao Li , Jinghua Zhu","doi":"10.1016/j.neunet.2024.106914","DOIUrl":"10.1016/j.neunet.2024.106914","url":null,"abstract":"<div><div>Medical image segmentation is essential for accurately representing tissues and organs in scans, improving diagnosis, guiding treatment, enabling quantitative analysis, and advancing AI-assisted healthcare. Organs and lesion areas in medical images have complex geometries and spatial relationships. Due to variations in the size and location of lesion areas, automatic segmentation faces significant challenges. While Convolutional Neural Networks (CNNs) and Transformers have proven effective in segmentation task, they still possess inherent limitations. Because these models treat images as regular grids or sequences of patches, they struggle to learn the geometric features of an image, which are essential for capturing irregularities and subtle details. In this paper we propose a novel segmentation model, MSGH, which utilizes Graph Neural Network (GNN) to fully exploit geometric representation for guiding image segmentation. In MSGH, we combine multi-scale features from Pyramid Feature and Graph Feature branches to facilitate information exchange across different networks. We also leverage graph contrastive representation learning to extract features through self-supervised learning to mitigate the impact of category imbalance in medical images. Moreover, we optimize the decoder by integrating Transformer to enhance the model’s capability in restoring the intricate image details feature. We conducted a comprehensive experimental study on ACDC, Synapse and BraTS datasets to validate the effectiveness and efficiency of MSGH. Our method achieved an improvement of 2.56–13.41%, 1.04–5.11% and 1.77–3.35% of dice on the three segmentation tasks respectively. The results demonstrate that our model consistently performs well compared with state-of-the-art models. The source code is accessible at <span><span>https://github.com/Dorothywujie/MSGH</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"182 ","pages":"Article 106914"},"PeriodicalIF":6.0,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142723737","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2024-11-23DOI: 10.1016/j.neunet.2024.106936
Xiaofeng Chen , Dongyuan Lin , Zhongshan Li , Weikai Li
{"title":"Iterative neural networks for improving memory capacity","authors":"Xiaofeng Chen , Dongyuan Lin , Zhongshan Li , Weikai Li","doi":"10.1016/j.neunet.2024.106936","DOIUrl":"10.1016/j.neunet.2024.106936","url":null,"abstract":"<div><div>In recent years, the problem of the multistability of neural networks has been studied extensively. From the research results obtained, the number of stable equilibrium points depends only on a power form of the network dimension. However, in practical applications, the number of stable equilibrium points needed is often not expressed in power form. Therefore, can we determine an appropriate activation function so that the neural network has exactly the required number of stable equilibrium points? This paper provides a new way to study this problem by means of an iteration method. The necessary activation function is constructed by an appropriate iteration method, and the neural network model is established. Based on the mathematical theories of matrix analysis and functional analysis and on the inequality method, the number and distribution of the network equilibrium points are determined by dividing the state space reasonably, and some multistability criteria that are related to the number of iterations and are independent of the network dimension are established.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"182 ","pages":"Article 106936"},"PeriodicalIF":6.0,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142723739","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2024-11-23DOI: 10.1016/j.neunet.2024.106926
Mallika Boyapati , Ramazan Aygun
{"title":"BalancerGNN: Balancer Graph Neural Networks for imbalanced datasets: A case study on fraud detection","authors":"Mallika Boyapati , Ramazan Aygun","doi":"10.1016/j.neunet.2024.106926","DOIUrl":"10.1016/j.neunet.2024.106926","url":null,"abstract":"<div><div>Fraud detection for imbalanced datasets is challenging due to machine learning models inclination to learn the majority class. Imbalance in fraud detection datasets affects how graphs are built, an important step in many Graph Neural Networks (GNNs). In this paper, we introduce our <em>BalancerGNN</em> framework to tackle with imbalanced datasets and show its effectiveness on fraud detection. Our framework has three major components: (i) node construction with feature representations, (ii) graph construction using balanced neighbor sampling, and (iii) GNN training using balanced training batches leveraging a custom loss function with multiple components. For node construction, we have introduced (i) Graph-based Variable Clustering (GVC) to optimize feature selection and remove redundancies by analyzing multi-collinearity and (ii) Encoder–Decoder based Dimensionality Reduction (EDDR) using transformer-based techniques to reduce feature dimensions while keeping important information intact about textual embeddings. Our experiments on Medicare, Equifax, IEEE, and auto insurance fraud datasets highlight the importance of node construction with features representations. BalancerGNN trained with balanced batches consistently outperforms other methods, showing strong abilities in identifying fraud cases, with sensitivity rates ranging from 72.87% to 81.23% across datasets while balancing specificity. Additionally, BalancerGNN achieves impressive accuracy rates, ranging from 73.99% to 94.28%. These outcomes underscore the crucial role of graph representation and neighbor sampling techniques in optimizing BalancerGNN for fraud detection models in real-world applications.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"182 ","pages":"Article 106926"},"PeriodicalIF":6.0,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142743441","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An object detection-based model for automated screening of stem-cells senescence during drug screening.","authors":"Yu Ren, Youyi Song, Mingzhu Li, Liangge He, Chunlun Xiao, Peng Yang, Yongtao Zhang, Cheng Zhao, Tianfu Wang, Guangqian Zhou, Baiying Lei","doi":"10.1016/j.neunet.2024.106940","DOIUrl":"https://doi.org/10.1016/j.neunet.2024.106940","url":null,"abstract":"<p><p>Deep learning-based cell senescence detection is crucial for accurate quantitative analysis of senescence assessment. However, senescent cells are small in size and have little differences in appearance and shape in different states, which leads to insensitivity problems such as missed and false detection. In addition, complex intelligent models are not conducive to clinical application. Therefore, to solve the above problems, we proposed a Faster Region Convolutional Neural Network (Faster R-CNN) detection model with Swin Transformer (Swin-T) and group normalization (GN), called STGF R-CNN, for the detection of different senescent cells to achieve quantification assessment of induced pluripotent stem cell-derived mesenchymal stem cells (iP-MSCs) senescence. Specifically, to enhance the representation learning ability of the network, Swin-T with a hierarchical structure was constructed. It utilizes a local window attention mechanism to capture features of different scales and levels. In addition, the GN strategy is adopted to achieve a lightweight model. To verify the effectiveness of the STGF R-CNN, a cell senescence dataset, the iP-MSCs dataset, was constructed, and a series of experiments were conducted. Experiment results show that it has the advantage of high senescent detection accuracy, mean Average Precision (mAP) is 0.835, Params is 46.06M, and FLOPs is 95.62G, which significantly reduces senescent assessment time from 12 h to less than 1 s. The STGF R-CNN has advantages over existing cell senescence detection methods, providing potential for anti-senescent drug screening. Our code is available at https://github.com/RY-97/STGF-R-CNN.</p>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"183 ","pages":"106940"},"PeriodicalIF":6.0,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142781696","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2024-11-23DOI: 10.1016/j.neunet.2024.106928
Yi Yan, Ercan Engin Kuruoglu
{"title":"Binarized Simplicial Convolutional Neural Networks","authors":"Yi Yan, Ercan Engin Kuruoglu","doi":"10.1016/j.neunet.2024.106928","DOIUrl":"10.1016/j.neunet.2024.106928","url":null,"abstract":"<div><div>Graph Neural Networks have the limitation of processing features solely on graph nodes, neglecting data on high-dimensional structures such as edges and triangles. Simplicial Convolutional Neural Networks (SCNN) represent high-order structures using simplicial complexes to break this limitation but still lack time efficiency. In this paper, a novel neural network architecture named Binarized Simplicial Convolutional Neural Networks (Bi-SCNN) is proposed based on the combination of simplicial convolution with a weighted binary-sign forward propagation strategy. The utilization of the Hodge Laplacian on a weighted binary-sign forward propagation enables Bi-SCNN to efficiently and effectively represent simplicial features with higher-order structures, surpassing the capabilities of traditional graph node representations. The Bi-SCNN achieves reduced model complexity compared to previous SSCN variants through binarization and normalization, also serving as intrinsic nonlinearities of Bi-SCNN; this enables Bi-SCNN to shorten the execution time without compromising prediction performance and makes Bi-SCNN less prone to over-smoothing. Experimenting with real-world citation and ocean-drifter data confirmed that our proposed Bi-SCNN is efficient and accurate.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"183 ","pages":"Article 106928"},"PeriodicalIF":6.0,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142757042","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2024-11-23DOI: 10.1016/j.neunet.2024.106838
Jinhua Lin , Xin Li , Lin Ma , Bowen Ren , Xiangdong Hao
{"title":"Separable integral neural networks","authors":"Jinhua Lin , Xin Li , Lin Ma , Bowen Ren , Xiangdong Hao","doi":"10.1016/j.neunet.2024.106838","DOIUrl":"10.1016/j.neunet.2024.106838","url":null,"abstract":"<div><div>Integral neural networks adopt continuous integral operators instead of conventional discrete convolutional operations to perform deep learning tasks. As this integral operator is the continuous representation of the regular convolutional operation, it is not suitable for representing the separable convolutional operations widely deployed on mobile devices. To address this issue, a separable integral layer composed of a depth-wise integral operator and a point-wise integral operator is proposed in this paper to represent discrete depth-wise and point-wise convolutional operations in continuous manner. According to the fabric units of five classical convolutional neural networks(NIN, VGG11, GoogleNet, ResNet18, ResNet50), we design five kinds of separable integral blocks(SIBs) to encapsulate separable integral layers in different manner. Using the proposed SIBs as basic blocks, a family of lightweight separable integral neural networks(SINNs) are constructed and deployed on resource-constrained mobile devices. SINNs have the characteristics of integral neural networks, i.e., performing structural pruning without fine-tuning, and also inherit the advantages of separable convolutional operations, i.e., reducing the computational cost while keeping a competitive performance. The experimental results show that SINNs achieve the similar performance with the state-of-the-art integral neural networks(INNs), while reducing the computational cost to up to 1/1.79 times that of INN(1.74× fewer parameters than INN using ResNet101 backbone framework) on ImageNet dataset. The code will be released at <span><span><em>https://github.com/ljh3832-ccut/SINN</em></span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"182 ","pages":"Article 106838"},"PeriodicalIF":6.0,"publicationDate":"2024-11-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142743440","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Unifying complete and incomplete multi-view clustering through an information-theoretic generative model","authors":"Yanghang Zheng , Guoxu Zhou , Haonan Huang , Xintao Luo , Zhenhao Huang , Qibin Zhao","doi":"10.1016/j.neunet.2024.106901","DOIUrl":"10.1016/j.neunet.2024.106901","url":null,"abstract":"<div><div>Recently, Incomplete Multi-View Clustering (IMVC) has become a rapidly growing research topic, driven by the prevalent issue of incomplete data in real-world applications. Although many approaches have been proposed to address this challenge, most methods did not provide a clear explanation of the learning process for recovery. Moreover, most of them only considered the inter-view relationships, without taking into account the relationships between samples. The influence of irrelevant information is usually ignored, which has prevented them from achieving optimal performance. To tackle the aforementioned issues, we aim at unifying compLete and incOmplete multi-view clusterinG through an Information-theoretiC generative model (LOGIC). Specifically, we have defined three principles based on information theory: comprehensiveness, consensus, and compressibility. We first explain that the essence of learning to recover missing views is to maximize the mutual information between the common representation and the data from each view. Secondly, we leverage the consensus principle to maximize the mutual information between view distributions to uncover the associations between different samples. Finally, guided by the principle of compressibility, we remove as much task-irrelevant information as possible to ensure that the common representation effectively extracts semantic information. Furthermore, it can serve as a plug-and-play missing-data recovery module for multi-view clustering models. Through extensive empirical studies, we have demonstrated the effectiveness of our approach in generating missing views. In clustering tasks, our method consistently outperforms state-of-the-art (SOTA) techniques in terms of accuracy, normalized mutual information and purity, showcasing its superiority in both recovery and clustering performance.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"182 ","pages":"Article 106901"},"PeriodicalIF":6.0,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142723736","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"I<sup>2</sup>HGNN: Iterative Interpretable HyperGraph Neural Network for semi-supervised classification.","authors":"Hongwei Zhang, Saizhuo Wang, Zixin Hu, Yuan Qi, Zengfeng Huang, Jian Guo","doi":"10.1016/j.neunet.2024.106929","DOIUrl":"https://doi.org/10.1016/j.neunet.2024.106929","url":null,"abstract":"<p><p>Learning on hypergraphs has garnered significant attention recently due to their ability to effectively represent complex higher-order interactions among multiple entities compared to conventional graphs. Nevertheless, the majority of existing methods are direct extensions of graph neural networks, and they exhibit noteworthy limitations. Specifically, most of these approaches primarily rely on either the Laplacian matrix with information distortion or heuristic message passing techniques. The former tends to escalate algorithmic complexity, while the latter lacks a solid theoretical foundation. To address these limitations, we propose a novel hypergraph neural network named I<sup>2</sup>HGNN, which is grounded in an energy minimization function formulated for hypergraphs. Our analysis reveals that propagation layers align well with the message-passing paradigm in the context of hypergraphs. I<sup>2</sup>HGNN achieves a favorable trade-off between performance and interpretability. Furthermore, it effectively balances the significance of node features and hypergraph topology across a diverse range of datasets. We conducted extensive experiments on 15 datasets, and the results highlight the superior performance of I<sup>2</sup>HGNN in the task of hypergraph node classification across nearly all benchmarking datasets.</p>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"183 ","pages":"106929"},"PeriodicalIF":6.0,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142781698","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2024-11-22DOI: 10.1016/j.neunet.2024.106927
Zhikui Chen, Lifang Li, Xu Zhang, Han Wang
{"title":"Deep graph clustering via aligning representation learning","authors":"Zhikui Chen, Lifang Li, Xu Zhang, Han Wang","doi":"10.1016/j.neunet.2024.106927","DOIUrl":"10.1016/j.neunet.2024.106927","url":null,"abstract":"<div><div>Deep graph clustering is a fundamental yet challenging task for graph data analysis. Recent efforts have witnessed significant success in combining autoencoder and graph convolutional network to explore graph-structured data. However, we observe that these approaches tend to map different nodes into the same representation, thus resulting in less discriminative node feature representation and limited clustering performance. Although some contrastive graph clustering methods alleviate the problem, they heavily depend on the carefully selected data augmentations, which greatly limits the capability of contrastive learning. Otherwise, they fail to consider the self-consistency between node representations and cluster assignments, thus affecting the clustering performance. To solve these issues, we propose a novel contrastive deep graph clustering method termed Aligning Representation Learning Network (ARLN). Specifically, we utilize contrastive learning between an autoencoder and a graph autoencoder to avoid conducting complex data augmentations. Moreover, we introduce an instance contrastive module and a feature contrastive module for consensus representation learning. Such modules are able to learn a discriminative node representation via contrastive learning. In addition, we design a novel assignment probability contrastive module to maintain the self-consistency between node representations and cluster assignments. Extensive experimental results on three benchmark datasets show the superiority of the proposed ARLN against the existing state-of-the-art deep graph clustering methods.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"183 ","pages":"Article 106927"},"PeriodicalIF":6.0,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142744232","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2024-11-22DOI: 10.1016/j.neunet.2024.106916
Yilin Chen , Yingnan Pan , Zhechen Zhu
{"title":"Finite-time optimal control for MMCPS via a novel preassigned-time performance approach","authors":"Yilin Chen , Yingnan Pan , Zhechen Zhu","doi":"10.1016/j.neunet.2024.106916","DOIUrl":"10.1016/j.neunet.2024.106916","url":null,"abstract":"<div><div>This paper studies the finite-time optimal stabilization problem of the macro–micro composite positioning stage (MMCPS). The dynamic model of the MMCPS is established as an interconnected system according to the Newton’s second law. Different from existing MMCPS control schemes, the convergence time of errors generated by control algorithms and coupling effects in the positioning process of the MMCPS is limited to the specific range depending on the initial value of the system, which is crucial for ensuring the cooperative work of macro and micro components. Meanwhile, the reinforcement learning strategy based on actor–critic neural networks is used to optimize the controller performance while ensuring the propulsion force on voice coil motor (VCM) and vibration reduction force on piezoelectric element actuator. Furthermore, a novel preassigned-time performance function is designed to guarantee that the displacements of the VCM axis and stage can be limited to the preassigned area in the preassigned time, thereby reducing vibration amplitude. All signals of the MMCPS system are proven to be semi-global practical finite-time stable. Finally, some simulation results demonstrate the feasibility of the designed algorithm.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"182 ","pages":"Article 106916"},"PeriodicalIF":6.0,"publicationDate":"2024-11-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142723738","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}