Deep multi-kernel cell clustering for single-cell RNA sequencing data

Abstract

Although existing deep learning methods for single-cell RNA sequencing (scRNA-seq) data can handle high-dimensional data and extract complex features, they also have the issue that the learned representations do not consider the clustering structure, leading to difficulties in linear separation. To address this, we propose a deep multi-kernel cell clustering network for scRNA-seq data, termed scDMKC, to cluster scRNA-seq data by simultaneously optimizing the multi-kernel representation of scRNA-seq data and the cell partitioning, enhancing the cell clustering performance by leveraging the strengths of multi-kernel learning and deep learning in capturing linearly separable data structures. To effectively learn the multi-kernel representation of scRNA-seq data, we propose a multi-kernel representation learner that adaptively selects an appropriate combination of multiple kernels to map the hidden representations of the data into the kernel space, thereby capturing the underlying linearly separable structure of the scRNA-seq data. In order to jointly optimize the multi-kernel representation of scRNA-seq data and the cell partitioning, a ZINB-based self-supervised strategy is developed. This strategy not only enhances the linear separability of the learned kernel representations but also improves clustering performance. Extensive experiments on various real scRNA-seq datasets were conducted. The experimental results indicate that our proposed scDMKC model significantly outperforms most of the existing cell clustering methods.