SPACE: Spatially variable gene clustering adjusting for cell type effect for improved spatial domain detection.

Recent advances in spatial transcriptomics (ST) have significantly deepened our understanding of biology. A primary focus in ST analysis is to identify spatially variable genes (SVGs) which are crucial for downstream tasks like spatial domain detection. Spatial domains reflect underlying tissue architecture and distinct biological processes. Traditional methods often use a set number of top SVGs for this purpose, and embedding these SVGs simultaneously can confound unrelated spatial signals, dilute weaker patterns, leading to obscured latent structure. Instead, grouping SVGs and getting low-dimensional embedding within each group preserves specific patterns, reduces signal mixing, and enhances the detection of diverse structures. Furthermore, classifying SVGs is akin to identifying cell-type marker genes, offering valuable biological insights. The challenge lies in accurately categorizing SVGs into relevant clusters, aggravated by the absence of prior knowledge regarding the number and spatial gene patterns. Here, we propose SPACE, a framework that classifies SVGs based on their spatial patterns by adjusting for shared cell-type confounding effects, to improve spatial domain detection. This method does not require prior knowledge of gene cluster numbers, spatial patterns, or cell type information. Both simulation and real data analyses demonstrate that SPACE is an efficient and promising tool for ST analysis.