RGAST: Relational Graph Attention Network for Spatial Transcriptome Analysis

Recent advancements in spatially resolved transcriptomics have provided a powerful means to comprehensively capture gene expression patterns while preserving the spatial context of the tissue microenvironment. Accurately deciphering the spatial context of spots within a tissue necessitates the careful utilization of their spatial information, which in turn requires feature extraction from complex and detailed spatial patterns. In this study, we present RGAST (Relational Graph Attention network for Spatial Transcriptome analysis), a framework designed to learn low-dimensional representations of spatial transcriptome (ST) data. RGAST is the first to consider gene expression similarity and spatial neighbor relationships simultaneously in constructing a heterogeneous graph network in ST analysis. We further introduce a cross-attention mechanism to provide a more comprehensive and adaptive representation of spatial transcriptome data. We validate the effectiveness of RGAST in different downstream tasks using diverse spatial transcriptomics datasets obtained from different platforms with varying spatial resolutions. Our results demonstrate that RGAST enhances spatial domain identification accuracy by approximately 10% compared to the second method in 10X Visium DLPFC dataset. Furthermore, RGAST facilitates the discovery of spatially variable genes, uncovers spatially resolved cell-cell interactions, enables more precise cell trajectory inference and reveals intricate 3D spatial patterns across multiple sections of ST data. Our RGAST method is available as a Python package on PyPI at https://pypi.org/project/RGAST, free for academic use, and the source code is openly available from our GitHub repository at https://github.com/GYQ-form/RGAST.